Mercurial > pygame

annotate docs/tut/camera/CameraIntro.html @ 3172:d1b367fc08bc tests-returncode

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use a return code of 1 when the tests fail.
author Thomas Kluyver <takowl@gmail.com>
date Wed, 09 Jan 2013 10:13:57 +0000
parents a7aa28b85cc2
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
1 <html><head><!--
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
2 TUTORIAL:Camera Module Introduction
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
3 -->
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
4 <title>Pygame Tutorials - Camera Module Introduction</title>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
5 </head>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
6 <body>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
7
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
8 <h1 align="center"><font size="-1">Pygame Tutorials</font><br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
9 Camera Module Introduction</h1>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
10
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
11 <h2 align="center">by Nirav Patel<br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
12 <font size="-1">nrp@eclecti.cc</font></h2>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
13
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
14 <h3 align="center">Revision 1.0, May 25th, 2009</h3>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
15 <br><br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
16
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
17 Pygame 1.9 comes with support for interfacing cameras, allowing you to capture
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
18 still images, watch live streams, and do some simple computer vision. This
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
19 tutorial will cover all of those use cases, providing code samples you can base
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
20 your app or game on. You can refer to the <a href="../../ref/camera.html">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
21 reference documentation</a> for the full API.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
22 <br><br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
23
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
24 <i>Note: As of Pygame 1.9, the camera module offers native support for cameras
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
25 that use v4l2 on Linux. There is support for other platforms via Videocapture
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
26 or OpenCV, but this guide will focus on the native module. Most of the code
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
27 will be valid for other platforms, but certain things like controls will not
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
28 work. The module is also marked as <b>EXPERIMENTAL</b>, meaning the API could
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
29 change in subsequent versions.</i>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
30 <br><br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
31
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
32 <h3>Import and Init</h3>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
33 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
34 import pygame
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
35 import pygame.camera
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
36 from pygame.locals import *
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
37
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
38 pygame.init()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
39 pygame.camera.init()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
40 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
41 As the camera module is optional, it needs to be imported and initialized
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
42 manually as shown above.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
43 <br><br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
44
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
45 <h3>Capturing a Single Image</h3>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
46 Now we will go over the simplest case of opening a camera and capturing a frame
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
47 as a surface. In the below example, we assume that there is a camera at
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
48 /dev/video0 on the computer, and initialize it with a size of 640 by 480.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
49 The surface called image is whatever the camera was seeing when get_image() was
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
50 called.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
51 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
52 cam = pygame.camera.Camera("/dev/video0",(640,480))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
53 cam.start()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
54 image = cam.get_image()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
55 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
56 <h4>Listing Connected Cameras</h4>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
57 You may be wondering, what if we don't know the exact path of the camera?
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
58 We can ask the module to provide a list of cameras attached to the
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
59 computer and initialize the first camera in the list.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
60 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
61 camlist = pygame.camera.list_cameras()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
62 if camlist:
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
63 cam = pygame.caemra.Camera(camlist[0],(640,480))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
64 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
65 <h4>Using Camera Controls</h4>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
66 Most cameras support controls like flipping the image and changing brightness.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
67 set_controls() and get_controls() can be used at any point after using start().
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
68 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
69 cam.set_controls(hflip = True, vflip = False)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
70 print camera.get_controls()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
71 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
72 <br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
73 <h3>Capturing a Live Stream</h3>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
74 The rest of this tutorial will be based around capturing a live stream of
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
75 images. For this, we will be using the class below. As described, it will
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
76 simply blit a constant stream of camera frames to the screen, effectively
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
77 showing live video. It is basically what you would expect, looping get_image(),
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
78 blitting to the display surface, and flipping it. For performance reasons,
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
79 we will be supplying the camera with the same surface to use each time.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
80 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
81 class Capture(object):
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
82 def __init__(self):
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
83 self.size = (640,480)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
84 # create a display surface. standard pygame stuff
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
85 self.display = pygame.display.set_mode(self.size, 0)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
86
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
87 # this is the same as what we saw before
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
88 self.clist = pygame.camera.list_cameras()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
89 if not self.clist:
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
90 raise ValueError("Sorry, no cameras detected.")
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
91 self.cam = pygame.camera.Camera(self.clist[0], self.size)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
92 self.cam.start()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
93
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
94 # create a surface to capture to. for performance purposes
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
95 # bit depth is the same as that of the display surface.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
96 self.snapshot = pygame.surface.Surface(self.size, 0, self.display)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
97
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
98 def get_and_flip(self):
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
99 # if you don't want to tie the framerate to the camera, you can check
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
100 # if the camera has an image ready. note that while this works
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
101 # on most cameras, some will never return true.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
102 if self.cam.query_image():
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
103 self.snapshot = self.cam.get_image(self.snapshot)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
104
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
105 # blit it to the display surface. simple!
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
106 self.display.blit(self.snapshot, (0,0))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
107 pygame.display.flip()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
108
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
109 def main(self):
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
110 going = True
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
111 while going:
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
112 events = pygame.event.get()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
113 for e in events:
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
114 if e.type == QUIT or (e.type == KEYDOWN and e.key == K_ESCAPE):
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
115 # close the camera safely
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
116 self.cam.stop()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
117 going = False
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
118
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
119 self.get_and_flip()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
120 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
121 Since get_image() is a blocking call that could take quite a bit of time on a
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
122 slow camera, this example uses query_image() to see if the camera is ready.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
123 This allows you to separate the framerate of your game from that of your camera.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
124 It is also possible to have the camera capturing images in a separate thread,
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
125 for approximately the same performance gain, if you find that your camera does
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
126 not support the query_image() function correctly.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
127 <br><br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
128
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
129 <h3>Basic Computer Vision</h3>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
130 By using the camera, transform, and mask modules, pygame can do some basic
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
131 computer vision.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
132 <br>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
133
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
134 <h4>Colorspaces</h4>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
135 When initializing a camera, colorspace is an optional parameter, with 'RGB',
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
136 'YUV', and 'HSV' as the possible choices. YUV and HSV are both generally more
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
137 useful for computer vision than RGB, and allow you to more easily threshold by
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
138 color, something we will look at later in the tutorial.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
139 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
140 self.cam = pygame.camera.Camera(self.clist[0], self.size, "RGB")
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
141 <img src="rgb.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
142 self.cam = pygame.camera.Camera(self.clist[0], self.size, "YUV")
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
143 <img src="yuv.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
144 self.cam = pygame.camera.Camera(self.clist[0], self.size, "HSV")
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
145 <img src="hsv.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
146 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
147
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
148 <h4>Thresholding</h4>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
149 Using the threshold() function from the transform module, one can do simple
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
150 green screen like effects, or isolate specifically colored objects in a scene.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
151 In the below example, we threshold out just the green tree and make the rest
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
152 of the image black. Check the reference documentation for details on the
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
153 <a href="../../ref/transform.html#pygame.transform.threshold">threshold
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
154 function</a>.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
155 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
156 self.thresholded = pygame.surface.Surface(self.size, 0, self.display)
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
157 self.snapshot = self.cam.get_image(self.snapshot)
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
158 pygame.transform.threshold(self.thresholded,self.snapshot,(0,255,0),(90,170,170),(0,0,0),2)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
159 <img src="thresholded.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
160 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
161 Of course, this is only useful if you already know the exact color of the object
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
162 you are looking for. To get around this and make thresholding usable in the
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
163 real world, we need to add a calibration stage where we identify the color of an
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
164 object and use it to threshold against. We will be using the average_color()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
165 function of the transform module to do this. Below is an example calibration
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
166 function that you could loop until an event like a key press, and an image of
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
167 what it would look like. The color inside the box will be the one that is
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
168 used for the threshold. Note that we are using the HSV colorspace in the below
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
169 images.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
170 <pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
171 def calibrate(self):
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
172 # capture the image
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
173 self.snapshot = self.cam.get_image(self.snapshot)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
174 # blit it to the display surface
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
175 self.display.blit(self.snapshot, (0,0))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
176 # make a rect in the middle of the screen
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
177 crect = pygame.draw.rect(self.display, (255,0,0), (145,105,30,30), 4)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
178 # get the average color of the area inside the rect
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
179 self.ccolor = pygame.transform.average_color(self.snapshot, crect)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
180 # fill the upper left corner with that color
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
181 self.display.fill(self.ccolor, (0,0,50,50))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
182 pygame.display.flip()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
183 <img src="average.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
184 pygame.transform.threshold(self.thresholded,self.snapshot,self.ccolor,(30,30,30),(0,0,0),2)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
185 <img src="thresh.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
186 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
187 You can use the same idea to do a simple green screen/blue screen, by first
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
188 getting a background image and then thresholding against it. The below example
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
189 just has the camera pointed at a blank white wall in HSV colorspace.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
190 <pre>
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
191 def calibrate(self):
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
192 # capture a bunch of background images
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
193 bg = []
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
194 for i in range(0,5):
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
195 bg.append(self.cam.get_image(self.background))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
196 # average them down to one to get rid of some noise
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
197 pygame.transform.average_surfaces(bg,self.background)
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
198 # blit it to the display surface
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
199 self.display.blit(self.background, (0,0))
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
200 pygame.display.flip()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
201 <img src="background.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
202 pygame.transform.threshold(self.thresholded,self.snapshot,(0,255,0),(30,30,30),(0,0,0),1,self.background)
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
203 <img src="green.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
204 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
205
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
206 <h4>Using the Mask Module</h4>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
207 The stuff above is great if you just want to display images, but with the
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
208 <a href="../../ref/mask.html">mask module</a>, you can also use a camera as an
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
209 input device for a game. For example, going back to the example of
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
210 thresholding out a specific object, we can find the position of that object and
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
211 use it to control an on screen object.
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
212 <pre>
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
213 def get_and_flip(self):
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
214 self.snapshot = self.cam.get_image(self.snapshot)
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
215 # threshold against the color we got before
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
216 mask = pygame.mask.from_threshold(self.snapshot, self.ccolor, (30, 30, 30))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
217 self.display.blit(self.snapshot,(0,0))
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
218 # keep only the largest blob of that color
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
219 connected = mask.connected_component()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
220 # make sure the blob is big enough that it isn't just noise
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
221 if mask.count() > 100:
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
222 # find the center of the blob
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
223 coord = mask.centroid()
2102
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
224 # draw a circle with size variable on the size of the blob
a7aa28b85cc2 set svn properties on html files for easier merges and less hassle after running makeref.py
Lenard Lindstrom <len-l@telus.net>
parents: 1820
diff changeset
225 pygame.draw.circle(self.display, (0,255,0), coord, max(min(50,mask.count()/400),5))
1820
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
226 pygame.display.flip()
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
227 <img src="mask.jpg">
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
228 </pre>
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
229 This is just the most basic example. You can track multiple different colored
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
230 blobs, find the outlines of objects, have collision detection between real life
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
231 and in game objects, get the angle of an object to allow for even finer control,
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
232 and more. Have fun!
1ee1d2993444 camera module and computer vision tutorial
Nirav Patel <nrp@eclecti.cc>
parents:
diff changeset
233 </body></html>