Hi all,
I have been working on a python script to control the drone with an xbox 360 controller and take the video from the drone and overlay some stuff on it like sensor data for a HUD display. and then once the script ends to process the video and recognize people.
I'm having an issue where the script lags. I suspect because reading the camera is taking longer than I would like and slowing down the program. At 20 fps i have .05 seconds per loop, and sometimes reading the camera is taking rather longer that .05 seconds. Do you have any tips on speeding it up? I was thinking I might try some threading when reading the camera as per:https://www.pyimagesearch.com/2017/02/06/faster-video-file-fps-with-cv2-videocapture-and-opencv/
Here is a screenshot of some of the logging output:
image url)
and here's a screenshot of some profiling i did with cProfile. I haven't used this before and am not great with it, but the time.sleep part looks weird to me. I'm not sure where this is getting called but am suspicious of it too. Maybe the open cv reads are sleeping while its encoding or decoding...
here's the loop part of the code that loops while the camera is opened
while cap.isOpened():
loop_start_time = time.time()
now = time.time()
r, frame = cap.read()
logging.info("Reading frame took {} seconds".format(time.time() - now))
# incrementing sensor_data, only read non-real time
# sensors every 20th frame. I defined which to not check each loop
# based on what I wanted real time updates on
sensor_data = update_sensor_information_logger(drone, sensor_data)
if sensor_data_counter % 20 == 0:
sensor_data = update_non_real_time_info_logger(drone, sensor_data)
logging.info("Roll: {} Yaw: {} Pitch: {} Battery: {}".format(
sensor_data["gyro_angles"].ROLL, sensor_data["gyro_angles"].YAW,
sensor_data["gyro_angles"].PITCH,sensor_data["battery_percentage"]))
sensor_data_counter = 0
sensor_data_counter += 1
# refreshing data from joystick
pygame.event.get()
# sending joystick commands, only if drone is in flight
sensor_data = command_top_gun_logger(drone, joystick, sensor_data, exit_flag)
# overlaying HUD on the frame
frame = hud_display_logger(frame, width, height, sensor_data, font)
# reading the buttons
sensor_data = get_joystick_buttons_logger(sensor_data, joystick)
# button is take off
if sensor_data["button_A"] and not take_off:
drone.takeoff()
take_off = True
# button B is land
elif sensor_data["button_B"] and take_off:
drone.land()
take_off = False
# button X is kill camera, start post-processing
elif sensor_data["button_X"]:
exit_flag = True
# button Y is emergency stop
elif sensor_data["button_Y"]:
drone.emergency_stop()
take_off = False
break
if exit_flag:
break
# displaying the frame and writing it
now = time.time()
cv2.imshow("frame", frame)
out.write(frame)
logging.info("Writing/showing frame took {} seconds".format(time.time() - now))
logging.info("full loop took {} seconds".format(time.time() - loop_start_time))
