Mercurial > cpython-lru_cache-weakref

annotate Include/pyatomic.h @ 69778:07613ea46dbc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Merge: I was right, hardconding the localhost IP doesn't work in linux-vserver.
author R David Murray <rdmurray@bitdance.com>
date Mon, 02 May 2011 08:48:00 -0400
parents 62b61abd02b8
children f855f929bc48
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
66664
62b61abd02b8 Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents: 61184
diff changeset
1 #ifndef Py_LIMITED_API
61184
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
2 #ifndef Py_ATOMIC_H
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
3 #define Py_ATOMIC_H
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
4 /* XXX: When compilers start offering a stdatomic.h with lock-free
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
5 atomic_int and atomic_address types, include that here and rewrite
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
6 the atomic operations in terms of it. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
7
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
8 #include "dynamic_annotations.h"
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
9
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
10 #ifdef __cplusplus
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
11 extern "C" {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
12 #endif
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
13
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
14 /* This is modeled after the atomics interface from C1x, according to
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
15 * the draft at
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
16 * http://www.open-std.org/JTC1/SC22/wg14/www/docs/n1425.pdf.
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
17 * Operations and types are named the same except with a _Py_ prefix
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
18 * and have the same semantics.
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
19 *
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
20 * Beware, the implementations here are deep magic.
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
21 */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
22
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
23 typedef enum _Py_memory_order {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
24 _Py_memory_order_relaxed,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
25 _Py_memory_order_acquire,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
26 _Py_memory_order_release,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
27 _Py_memory_order_acq_rel,
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
28 _Py_memory_order_seq_cst
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
29 } _Py_memory_order;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
30
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
31 typedef struct _Py_atomic_address {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
32 void *_value;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
33 } _Py_atomic_address;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
34
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
35 typedef struct _Py_atomic_int {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
36 int _value;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
37 } _Py_atomic_int;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
38
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
39 /* Only support GCC (for expression statements) and x86 (for simple
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
40 * atomic semantics) for now */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
41 #if defined(__GNUC__) && (defined(__i386__) || defined(__amd64))
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
42
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
43 static __inline__ void
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
44 _Py_atomic_signal_fence(_Py_memory_order order)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
45 {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
46 if (order != _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
47 __asm__ volatile("":::"memory");
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
48 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
49
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
50 static __inline__ void
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
51 _Py_atomic_thread_fence(_Py_memory_order order)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
52 {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
53 if (order != _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
54 __asm__ volatile("mfence":::"memory");
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
55 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
56
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
57 /* Tell the race checker about this operation's effects. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
58 static __inline__ void
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
59 _Py_ANNOTATE_MEMORY_ORDER(const volatile void *address, _Py_memory_order order)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
60 {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
61 switch(order) {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
62 case _Py_memory_order_release:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
63 case _Py_memory_order_acq_rel:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
64 case _Py_memory_order_seq_cst:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
65 _Py_ANNOTATE_HAPPENS_BEFORE(address);
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
66 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
67 default:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
68 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
69 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
70 switch(order) {
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
71 case _Py_memory_order_acquire:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
72 case _Py_memory_order_acq_rel:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
73 case _Py_memory_order_seq_cst:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
74 _Py_ANNOTATE_HAPPENS_AFTER(address);
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
75 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
76 default:
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
77 break;
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
78 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
79 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
80
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
81 #define _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
82 __extension__ ({ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
83 __typeof__(ATOMIC_VAL) atomic_val = ATOMIC_VAL; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
84 __typeof__(atomic_val->_value) new_val = NEW_VAL;\
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
85 volatile __typeof__(new_val) *volatile_data = &atomic_val->_value; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
86 _Py_memory_order order = ORDER; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
87 _Py_ANNOTATE_MEMORY_ORDER(atomic_val, order); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
88 \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
89 /* Perform the operation. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
90 _Py_ANNOTATE_IGNORE_WRITES_BEGIN(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
91 switch(order) { \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
92 case _Py_memory_order_release: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
93 _Py_atomic_signal_fence(_Py_memory_order_release); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
94 /* fallthrough */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
95 case _Py_memory_order_relaxed: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
96 *volatile_data = new_val; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
97 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
98 \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
99 case _Py_memory_order_acquire: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
100 case _Py_memory_order_acq_rel: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
101 case _Py_memory_order_seq_cst: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
102 __asm__ volatile("xchg %0, %1" \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
103 : "+r"(new_val) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
104 : "m"(atomic_val->_value) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
105 : "memory"); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
106 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
107 } \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
108 _Py_ANNOTATE_IGNORE_WRITES_END(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
109 })
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
110
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
111 #define _Py_atomic_load_explicit(ATOMIC_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
112 __extension__ ({ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
113 __typeof__(ATOMIC_VAL) atomic_val = ATOMIC_VAL; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
114 __typeof__(atomic_val->_value) result; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
115 volatile __typeof__(result) *volatile_data = &atomic_val->_value; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
116 _Py_memory_order order = ORDER; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
117 _Py_ANNOTATE_MEMORY_ORDER(atomic_val, order); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
118 \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
119 /* Perform the operation. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
120 _Py_ANNOTATE_IGNORE_READS_BEGIN(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
121 switch(order) { \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
122 case _Py_memory_order_release: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
123 case _Py_memory_order_acq_rel: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
124 case _Py_memory_order_seq_cst: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
125 /* Loads on x86 are not releases by default, so need a */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
126 /* thread fence. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
127 _Py_atomic_thread_fence(_Py_memory_order_release); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
128 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
129 default: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
130 /* No fence */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
131 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
132 } \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
133 result = *volatile_data; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
134 switch(order) { \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
135 case _Py_memory_order_acquire: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
136 case _Py_memory_order_acq_rel: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
137 case _Py_memory_order_seq_cst: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
138 /* Loads on x86 are automatically acquire operations so */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
139 /* can get by with just a compiler fence. */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
140 _Py_atomic_signal_fence(_Py_memory_order_acquire); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
141 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
142 default: \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
143 /* No fence */ \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
144 break; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
145 } \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
146 _Py_ANNOTATE_IGNORE_READS_END(); \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
147 result; \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
148 })
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
149
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
150 #else /* !gcc x86 */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
151 /* Fall back to other compilers and processors by assuming that simple
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
152 volatile accesses are atomic. This is false, so people should port
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
153 this. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
154 #define _Py_atomic_signal_fence(/*memory_order*/ ORDER) ((void)0)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
155 #define _Py_atomic_thread_fence(/*memory_order*/ ORDER) ((void)0)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
156 #define _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
157 ((ATOMIC_VAL)->_value = NEW_VAL)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
158 #define _Py_atomic_load_explicit(ATOMIC_VAL, ORDER) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
159 ((ATOMIC_VAL)->_value)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
160
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
161 #endif /* !gcc x86 */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
162
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
163 /* Standardized shortcuts. */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
164 #define _Py_atomic_store(ATOMIC_VAL, NEW_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
165 _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, _Py_memory_order_seq_cst)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
166 #define _Py_atomic_load(ATOMIC_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
167 _Py_atomic_load_explicit(ATOMIC_VAL, _Py_memory_order_seq_cst)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
168
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
169 /* Python-local extensions */
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
170
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
171 #define _Py_atomic_store_relaxed(ATOMIC_VAL, NEW_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
172 _Py_atomic_store_explicit(ATOMIC_VAL, NEW_VAL, _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
173 #define _Py_atomic_load_relaxed(ATOMIC_VAL) \
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
174 _Py_atomic_load_explicit(ATOMIC_VAL, _Py_memory_order_relaxed)
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
175
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
176 #ifdef __cplusplus
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
177 }
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
178 #endif
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
179
4fd3ab5fa71d Make (most of) Python's tests pass under Thread Sanitizer.
Jeffrey Yasskin <jyasskin@gmail.com>
parents:
diff changeset
180 #endif /* Py_ATOMIC_H */
66664
62b61abd02b8 Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents: 61184
diff changeset
181 #endif /* Py_LIMITED_API */