Image enhancement for wireless webcam from X10, Igor Borovikov amateur astronomy

back to image processing	back to the main page
Wireless Webcam as a Test Case for Image Enhancement Techniques
Wireless Webcam Kit I got my wireless webcams from www.x10.com. The website offers various “super saver” packages. The set I’ve got included 3 wireless webcams with addressable power supply, video receiver, firecracker power switch and some other stuff. The wireless webcam kit works as follows. The user sets up cameras at different locations inside the house and powers them up via addressable power supply. The video receiver connects to the computer via USB converter. Also you need to plug in firecracker in one of the power outlets and attach firecracker transmitter to COM port of your PC. The package comes with software called “XRay Vision”. It consists of two components. The first one switches cameras by talking to the device in the back of your computer making it send radio signals to the firecracker switch. The switch sends signals via the electric wiring of the house to the addressable power supplies of cameras making them go on and off. When one of the cameras is active it sends its video signal to the receiver. The receiver is connected to you PC via video-to-USB converter and the video signal is delivered as usual webcam feed. Some Related Freeware Since both firecracker and USB input follow standard protocols it is possible to develop your own custom applications to control cameras and process incoming images. Scripting in Python is probably one of the best alternatives. Firecracker software is readily available from HomeSeer LLC as CM17 ActiveX control at no charge. You can call ActiveX control from Python using Python Win32 extensions. See Appendix 1 for the full source code of simple wrapper class for firecracker. It seems to work fine except for couple of idiosyncrasies. One of them is occasional power switch malfunction. I think that is coming from camera power supply overheating. When it is happening the camera does not want to switch off, it keeps sending the signal regardless and messes up video input. Physically unplugging problematic camera for a while helps. The other problem is occasional hanging up of the ActiveX control when it resets firecracker interface. I didn’t track down this problem yet. It’s easy to capture video in Python with vidcap module for win32 by Markus Gritsch. The interface is very simple; see example in Appendix 2. The image coming from video capture module is PIL object (PIL is Python Image Library). This object is all you need to start messing up with video input. The Raw Image The video signal from X10 wireless cameras is quite noisy. Even if the camera is close to the receiver still there is a lot of interference. If the camera is on the other floor or is not pointing at the receiver, the signal gets even worse.
	Besides video noise low light sensitivity affects performance of the regular XCam2 cameras quite a bit. With only scattered electric light in the room the picture is marginal. The image on the left is very dark and you can see a strip introduced by noisy video signal. In addition to spurious strips there is some rippling noise. It is more apparent on brighter images.
Pulling out anything useful from the imput like that is a good excersise in image enhancement. While doing that I wrote a simple Python module. The original version version was all-Ptyhon, but to imporve perfomrmance I ported the most calculation intense parts into C. Stacking Collecting a number of raw frames then stacking them allows to improve signal/noise ratio.
Let F_i(x,y) be the pixel (gray scale value) recorded in frame i at the image coordinates (x,y). Then stacked image consists of pixels averaged at the same coordinates over all images in the stack of N images: F_stacked(x,y)=Sum_(i=1...N) F_i(x,y)/N. For RGB images the same procedure should be run on all RGB components.
The raw frames in my case are 8 bit RGB images. Stacking them in 8-bit color space does not quite work because the result would still suffer from quantization artifacts. To address that I used matrix of floating point numbers to accumulate 128 raw frames. While straightforward stacking was reducing noise considerably, the result was still degraded by random strips of bad video signal. Fortunately bad signal strips can be easily detected and excluded from stack. The first version was rejecting complete frame if it would detect a bad strip in the image. Because of that up to 40 precent of frames would be rejected. To improve percent of used captured data I switched to stacking on per-strip basis. The usual stacking algorithm worked independently on each horizontal strip so the amount of rejected data was minimized more than twice. Dark Frame Subtraction Because of the thermal noise in CCD the image is not completely black even if the objective lens is covered. An image recorded with no incoming light is called dark frame. Subtracting dark frame from the images allows to remove or reduce significanlty the level of intrinsic noise of the camera. In the case of wireless cameras dark frames were affected by video signal interference. By stacking dark frames in the same fashion as normal frames I obtained some good approxmation to the dark frames for each of my cameras.
	A dark frame appears to be perfectly black. However Autolevels filter in Photoshop reveals that it is not. Only the half of the dark frame to the right was "autolevelled" for comaprison. Below is a histogram of the dark frame.
Histogram Equalization After stacking and extracting dark frame the image was still far from been usable. An apparently useful step in the following image enhancement could be full-scale histogram stretch (FSHS, aka Autolevels in Photoshop terminology). Yet the problem remained with too many dark areas and low contrast, even after some gamma correction. Histogram equalization allows to address that problem. The method goal is to equalize the number of pixels for each intensity range and produce a new image with the resulting histogram as close to the straigh horizontal tline as possible (the so called flat histogram).
Suppose we have a gray scale NxM image with FSHS (Autolevels) already applied. Let's assume also that the pixel values are floating point numbers normaized to [0...1]. Consider K equal length segments [h_i,h_i+1], i = 0,..., K-1, where h₀=0 and h_K=1, covering the entire range [0...1]. Let P_i be the number of pixels with values v in the range h_i < v < h_i+1and let n = NxM be the total number of pixels in the image. When we randomly pick a pixel from the image the number P_i /n represents the probability that we will select a pixel from the i-th segement. In the image with the equalized histogram this probability must be the same for all segements. To equalize this probabiliy we should transform pixels in each of the original segements to spread them more uniformly. The more pixels are in the original segement the more we should stretch the segment to reduce chances of picking a pixel from it. The segements with smaller P_ishould be compressed for the same reason. Overall the sum of all probabilities has to be equal 1. That leads to the following iterative formula for the new segment boundaries q_i : q₀ = 0, q₁ = P₁/n, q₂=(P₁+P₂)/n, q₃=(P₁+P₂+P₃)/n, ..., q_K=1. It is easy to see that indeed q_K=1, so the sum of probabilities is equal 1. Also, evidently the number of pixels in each of the new segments divided by the length of the segment is constant. The remaining step in equalization is to map pixels with values in the range h_i...h_i+1to the range q_i...q_i+1for all segements, which is an easy excercise.
I used 512 segments in the histogram equalization. It is twice more than 255 levels in the final 8-bit image, so using floating point numbers was essential. Since the floating point image was obtained from stacking of many 8-bit images, there was plenty information hidden in low brightness pixels. Usng 512 (or even more) segments helped to pulled that information and convert it into more useful range without false contouring. As always, a larger pixel depth (number of levels per color component) prevents from false contouring during histogram equalization.
	This fully processed image is the result of stacking of 128 images like the first one presented in this article. The dark frame was subtracted and histogram equalized.
On the image above the histogram equalization, besides enhancing useful details, also enhanced the residual noise. For aesthetic reasons the result of equalization was mixed 50-50 with the original image. The same trick was used when subtracting dark frame (the dark frame was quite imprecise anyway).
Appendix 1. Firecracker class in Python. (under construction)
Appendix 2. Sample Python code that captures n frames with vidcap. (under construction)
Appendix 3. Example of calling x10stack module. (under construction)
Appendix 4. Download x10stacker.dll, an extension module for Python. (under contsurction)
Literature [1] Handbook of CCD Astronomy, Steve B. Howell, Cambridge University Press, 2000, 164 pages. [2] Basic Gray-Level Image Processing, Alan C. Bovik in: Handbook of Image and Video Processing, Academic Press, 2000, p.21-36.

back to image processing	back to the main page

Wireless Webcam Kit

Some Related Freeware

Stacking