We discussed the OpenCV library basics in one of our previous articles – Getting Started with OpenCV in Python. In continuation with that, we discuss the GUI interactions and explore the powerful real-time user control capabilities of the OpenCV library in this article.
A GUI (Graphical User Interface) is a visual system that interacts with the user directly through inputs and outputs. OpenCV displays an image or a video through a rectangular window. It also receives inputs from the user through that window by creating trackbars and reading mouse movements and keyboard taps. Thus windows in OpenCV behave as individual and independent GUIs.
Keyboard Interactions
OpenCV can directly read keyboard inputs while executing its program and make decisions according to the input made. In the below example, we read an image and display it in a window. If the key ‘q’ is pressed on the keyboard, the window will be closed immediately. Else, if the key ‘s’ is pressed, the image will be saved to the disk, and the window will be closed. Rather, if the key ‘g’ is pressed, the image will be saved to the disk in grayscale, and the window will be closed. Else, if the key ‘t’ is pressed, the image will be saved to the disk with some predefined texts written on it, and the window will be closed. The choice of keys and the actions to be performed have no constraints and are up to the users’ creativity and requirements!
import cv2
# read a colour image from the working directory
img = cv2.imread('mimi.jpg')
img = cv2.resize(img,(320,240))
# display the original image
cv2.imshow('Original Image', img)
key = cv2.waitKey(0) & 0xFF
# KEYBOARD INTERACTIONS
if key == ord('q'):
cv2.destroyAllWindows()
elif key == ord('s'):
# save the image as such
cv2.imwrite('mimi_colour.jpg', img)
cv2.destroyAllWindows()
elif key == ord('g'):
# convert to grayscale and save it
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.imwrite('mimi_gray.jpg', gray)
cv2.destroyAllWindows()
elif key == ord('t'):
# write some text and save it
text_image = cv2.putText(img, 'Miracles of OpenCV', (10,30), cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,0),3)
cv2.imwrite('mimi_text.jpg', text_image)
cv2.destroyAllWindows()
The above codes are run four times repeatedly to explore all four key input options.
The saved images can be visualized in three separate windows using the following codes to realize how the keyboard interactions work.
# display the saved colour image
clr_img = cv2.imread('mimi_colour.jpg')
cv2.imshow('Colour Image', clr_img)
# display the saved grayscale image
gray_img = cv2.imread('mimi_gray.jpg')
cv2.imshow('Grayscale Image', gray_img)
# display the image with Text on it
text_img = cv2.imread('mimi_text.jpg')
cv2.imshow('Image with Text', text_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Creating and Reading a Trackbar
A Trackbar is a control system that takes user input directly through the GUI window. The below example code creates a Trackbar to do a switch between two different images. It takes an integer input either 0 or 1 and displays the corresponding image. The slider in the Trackbar must be manually shifted between 0 and 1.
# read two images
img_1 = cv2.imread('karsten_1.jpg', 1)
img_2 = cv2.imread('karsten_2.jpg', 1)
# create a window
cv2.namedWindow("My Pet")
# define a null callback function
def null(x):
pass
# create a trackbar
# arguments: trackbar_name, window_name, default_value, max_value, callback_fn
cv2.createTrackbar("Switch_Image", "My Pet", 0, 1, null)
while True:
# get Trackbar position
pos = cv2.getTrackbarPos("Switch_Image", "My Pet")
if pos == 0:
cv2.imshow("My Pet", img_1)
elif pos == 1:
cv2.imshow("My Pet", img_2)
key = cv2.waitKey(1) & 0xFF
# press 'q' to quit the window
if key == ord('q'):
break
cv2.destroyAllWindows()
BGR Colour Control Using Trackbars
Trackbars can be used to fine-tune the colour selection in real-time for creative applications. In the following example code, the three colour channels Blue, Green and Red are controlled by three Trackbars independently to arrive at the final BGR colour of users’ interest or requirement. The minimum value for each channel is 0 and the maximum value is 255. Trackbars read slider positions as integers only. Hence each Trackbar can be positioned at 256 slider positions.
# BGR Control using Trackbars
import numpy as np
# create a black image
img = np.zeros([200,350,3], np.uint8)
cv2.namedWindow('BGR')
# define a null callback function for Trackbar
def null(x):
pass
# create three trackbars for B, G and R
# arguments: trackbar_name, window_name, default_value, max_value, callback_fn
cv2.createTrackbar("B", "BGR", 0, 255, null)
cv2.createTrackbar("G", "BGR", 0, 255, null)
cv2.createTrackbar("R", "BGR", 0, 255, null)
while True:
# read the Trackbar positions
b = cv2.getTrackbarPos('B','BGR')
g = cv2.getTrackbarPos('G','BGR')
r = cv2.getTrackbarPos('R','BGR')
# change the image colour to Trackbar positions
img[:] = [b,g,r]
# display trackbars and image
cv2.imshow('BGR', img)
key = cv2.waitKey(1) & 0xFF
if key == ord('q'):
break
cv2.destroyAllWindows()
Sample window displays for various Trackbar positions:
HSV Colour Control Using Trackbars
Colour control can be made in a human-understandable way using the HSV colour space. HSV is the abbreviation for Hue-Saturation-Value. A HSV colour space is represented cylindrical in shape. Hue is the variation in core colour that varies circumferentially in the cylinder having values ranging from 0 degree to 180 degrees in OpenCV. Saturation defines the richness of the colour, which is the radial variation within the colour having values ranging from 0 to 255. Value defines the brightness of the colour, which is the axial variation having values ranging from 0 to 255.
# HSV Control using Trackbars
# read a colourful image
img = cv2.imread('claudio.jpg')
img = cv2.resize(img, (320,280))
# convert BGR image to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# define a null callback function for Trackbar
def null(x):
pass
# create six trackbars for H, S and V - lower and higher masking limits
cv2.namedWindow('HSV')
# arguments: trackbar_name, window_name, default_value, max_value, callback_fn
cv2.createTrackbar("HL", "HSV", 0, 180, null)
cv2.createTrackbar("HH", "HSV", 180, 180, null)
cv2.createTrackbar("SL", "HSV", 0, 255, null)
cv2.createTrackbar("SH", "HSV", 255, 255, null)
cv2.createTrackbar("VL", "HSV", 0, 255, null)
cv2.createTrackbar("VH", "HSV", 255, 255, null)
while True:
# read the Trackbar positions
hl = cv2.getTrackbarPos('HL','HSV')
hh = cv2.getTrackbarPos('HH','HSV')
sl = cv2.getTrackbarPos('SL','HSV')
sh = cv2.getTrackbarPos('SH','HSV')
vl = cv2.getTrackbarPos('VL','HSV')
vh = cv2.getTrackbarPos('VH','HSV')
# create a manually controlled mask
# arguments: hsv_image, lower_trackbars, higher_trackbars
mask = cv2.inRange(hsv, np.array([hl, sl, vl]), np.array([hh, sh, vh]))
# derive masked image using bitwise_and method
final = cv2.bitwise_and(img, img, mask=mask)
# display image, mask and masked_image
cv2.imshow('Original', img)
cv2.imshow('Mask', mask)
cv2.imshow('Masked Image', final)
key = cv2.waitKey(1) & 0xFF
if key == ord('q'):
break
cv2.destroyAllWindows()
With advanced image processing methods of OpenCV, the HSV analysis yields better image processing abilities. Learn more about HSV colour space here.
Interactions with Mouse Movements
Tracing mouse movements and mouse button clicks is a powerful tool available in the OpenCV library. OpenCV methods can trace left button down/up, right button down/up, left button double click, mouse movement, etc. This makes the library highly interactive with user controls through mouse. In the following example, the mouse left button’s down action is used to read the colour where it is clicked. With the observed colour, a colour-filled rectangular box is drawn on another image.
# read a colourful image
img_1 = cv2.imread('balls.jpg')
img_1 = cv2.resize(img_1, (320,210))
# display the image
cv2.imshow('Colour', img_1)
# read another image to display clicked colour
img_2 = cv2.imread('julian.jpg')
img_2 = cv2.resize(img_2, (320,210))
def Mouse_Event(event, x, y, flags, param):
if event == cv2.EVENT_LBUTTONDOWN:
# read colours at left clicked point
b = img[x,y,0]
g = img[x,y,1]
r = img[x,y,2]
# change the colour of a portion of image
coloured = img_2.copy()
coloured[-50:-1, 0:320, :] = [b,g,r]
coloured = cv2.putText(coloured, 'Colourful Holi!', (40,190), cv2.FONT_HERSHEY_PLAIN, 2, (0,255,255), 2)
cv2.imshow('Clicked Colour', coloured)
# set Mouse Callback method
cv2.setMouseCallback('Colour', Mouse_Event)
cv2.waitKey(0)
cv2.destroyAllWindows()
Wrapping up
In this article, we discussed the GUI features of the OpenCV library and explored how they contribute to real-time interactions with the users. We discussed Keyboard interactions, Mouse interactions and Trackbars. We learnt how to use Trackbars in different use cases, including BGR colour analysis, HSV colour analysis and masking an image based on colours.
Note: The images used in this hands-on tutorial are open source images that need no reuse license.
