Mercurial > cpython-lru_cache-weakref

annotate Include/pyatomic.h @ 62799:dcc2e8bf64cf

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix: Issue9091 Minor documentation clarification.
author Senthil Kumaran <orsenthil@gmail.com>
date Mon, 05 Jul 2010 11:41:42 +0000
parents 4fd3ab5fa71d
children 62b61abd02b8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
61184
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
1 #ifndef Py_ATOMIC_H
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
2 #define Py_ATOMIC_H
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
3 /* XXX: When compilers start offering a stdatomic.h with lock-free
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
4 atomic_int and atomic_address types, include that here and rewrite
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
5 the atomic operations in terms of it. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
6
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
7 #include "dynamic_annotations.h"
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
8
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
9 #ifdef __cplusplus
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
10 extern "C" {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
11 #endif
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
12
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
13 /* This is modeled after the atomics interface from C1x, according to
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
14 * the draft at
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
15 * http://www.open-std.org/JTC1/SC22/wg14/www/docs/n1425.pdf.
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
16 * Operations and types are named the same except with a _Py_ prefix
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
17 * and have the same semantics.
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
18 *
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
19 * Beware, the implementations here are deep magic.
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
20 */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
21
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
22 typedef enum _Py_memory_order {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
23 _Py_memory_order_relaxed,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
24 _Py_memory_order_acquire,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
25 _Py_memory_order_release,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
26 _Py_memory_order_acq_rel,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
27 _Py_memory_order_seq_cst
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
28 } _Py_memory_order;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
29
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
30 typedef struct _Py_atomic_address {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
31 void *_value;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
32 } _Py_atomic_address;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
33
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
34 typedef struct _Py_atomic_int {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
35 int _value;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
36 } _Py_atomic_int;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
37
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
38 /* Only support GCC (for expression statements) and x86 (for simple
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
39 * atomic semantics) for now */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
40 #if defined(__GNUC__) && (defined(__i386__) || defined(__amd64))
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
41
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
42 static __inline__ void
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
43 _Py_atomic_signal_fence(_Py_memory_order order)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
44 {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
45 if (order != _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
46 __asm__ volatile("":::"memory");
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
47 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
48
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
49 static __inline__ void
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
50 _Py_atomic_thread_fence(_Py_memory_order order)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
51 {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
52 if (order != _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
53 __asm__ volatile("mfence":::"memory");
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
54 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
55
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
56 /* Tell the race checker about this operation's effects. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
57 static __inline__ void
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
58 _Py_ANNOTATE_MEMORY_ORDER(const volatile void *address, _Py_memory_order order)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
59 {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
60 switch(order) {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
61 case _Py_memory_order_release:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
62 case _Py_memory_order_acq_rel:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
63 case _Py_memory_order_seq_cst:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
64 _Py_ANNOTATE_HAPPENS_BEFORE(address);
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
65 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
66 default:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
67 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
68 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
69 switch(order) {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
70 case _Py_memory_order_acquire:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
71 case _Py_memory_order_acq_rel:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
72 case _Py_memory_order_seq_cst:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
73 _Py_ANNOTATE_HAPPENS_AFTER(address);
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
74 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
75 default:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
76 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
77 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
78 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
79
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
80 #define _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
81 __extension__ ({ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
82 __typeof__(ATOMIC_VAL) atomic_val = ATOMIC_VAL; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
83 __typeof__(atomic_val->_value) new_val = NEW_VAL;\
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
84 volatile __typeof__(new_val) *volatile_data = &atomic_val->_value; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
85 _Py_memory_order order = ORDER; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
86 _Py_ANNOTATE_MEMORY_ORDER(atomic_val, order); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
87 \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
88 /* Perform the operation. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
89 _Py_ANNOTATE_IGNORE_WRITES_BEGIN(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
90 switch(order) { \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
91 case _Py_memory_order_release: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
92 _Py_atomic_signal_fence(_Py_memory_order_release); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
93 /* fallthrough */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
94 case _Py_memory_order_relaxed: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
95 *volatile_data = new_val; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
96 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
97 \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
98 case _Py_memory_order_acquire: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
99 case _Py_memory_order_acq_rel: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
100 case _Py_memory_order_seq_cst: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
101 __asm__ volatile("xchg %0, %1" \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
102 : "+r"(new_val) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
103 : "m"(atomic_val->_value) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
104 : "memory"); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
105 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
106 } \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
107 _Py_ANNOTATE_IGNORE_WRITES_END(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
108 })
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
109
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
110 #define _Py_atomic_load_explicit(ATOMIC_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
111 __extension__ ({ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
112 __typeof__(ATOMIC_VAL) atomic_val = ATOMIC_VAL; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
113 __typeof__(atomic_val->_value) result; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
114 volatile __typeof__(result) *volatile_data = &atomic_val->_value; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
115 _Py_memory_order order = ORDER; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
116 _Py_ANNOTATE_MEMORY_ORDER(atomic_val, order); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
117 \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
118 /* Perform the operation. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
119 _Py_ANNOTATE_IGNORE_READS_BEGIN(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
120 switch(order) { \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
121 case _Py_memory_order_release: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
122 case _Py_memory_order_acq_rel: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
123 case _Py_memory_order_seq_cst: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
124 /* Loads on x86 are not releases by default, so need a */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
125 /* thread fence. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
126 _Py_atomic_thread_fence(_Py_memory_order_release); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
127 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
128 default: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
129 /* No fence */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
130 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
131 } \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
132 result = *volatile_data; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
133 switch(order) { \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
134 case _Py_memory_order_acquire: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
135 case _Py_memory_order_acq_rel: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
136 case _Py_memory_order_seq_cst: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
137 /* Loads on x86 are automatically acquire operations so */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
138 /* can get by with just a compiler fence. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
139 _Py_atomic_signal_fence(_Py_memory_order_acquire); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
140 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
141 default: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
142 /* No fence */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
143 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
144 } \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
145 _Py_ANNOTATE_IGNORE_READS_END(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
146 result; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
147 })
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
148
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
149 #else /* !gcc x86 */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
150 /* Fall back to other compilers and processors by assuming that simple
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
151 volatile accesses are atomic. This is false, so people should port
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
152 this. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
153 #define _Py_atomic_signal_fence(/*memory_order*/ ORDER) ((void)0)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
154 #define _Py_atomic_thread_fence(/*memory_order*/ ORDER) ((void)0)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
155 #define _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
156 ((ATOMIC_VAL)->_value = NEW_VAL)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
157 #define _Py_atomic_load_explicit(ATOMIC_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
158 ((ATOMIC_VAL)->_value)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
159
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
160 #endif /* !gcc x86 */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
161
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
162 /* Standardized shortcuts. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
163 #define _Py_atomic_store(ATOMIC_VAL, NEW_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
164 _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, _Py_memory_order_seq_cst)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
165 #define _Py_atomic_load(ATOMIC_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
166 _Py_atomic_load_explicit(ATOMIC_VAL, _Py_memory_order_seq_cst)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
167
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
168 /* Python-local extensions */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
169
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
170 #define _Py_atomic_store_relaxed(ATOMIC_VAL, NEW_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
171 _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
172 #define _Py_atomic_load_relaxed(ATOMIC_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
173 _Py_atomic_load_explicit(ATOMIC_VAL, _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
174
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
175 #ifdef __cplusplus
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
176 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
177 #endif
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
178
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
179 #endif /* Py_ATOMIC_H */