We will use Python programming to send commands to the robotic arm. > In this exercise, the Ned2 robotic arm will perform a pick-and-place operation using text recognition.

Ned2 – Pick & Place powered by OCR (EasyOCR + PyNiryo2)

Estimated duration: ~1h30 (Setup 10 min · OCR Intro 10 min · Guided project & challenges 1h)

Contents

1. Prerequisites
2. Technical environment
3. Setup (safety & connections)
4. Chapter 1 – Setup (10 min)
5. Chapter 2 – Introduction to OCR & image-based control (10 min)
6. Chapter 3 – Your turn! (1h)
7. Solution – Full code (with user feedback)
8. Chapter 4 (extra) – Libraries used

Prerequisites

• Python: basic syntax, data and control structures (list, dictionary, if, for, while), function calls, and some familiarity with PyNiryo2 (keep the documentation open if needed). Ability to install additional libraries if required.
• Ned2 & NiryoStudio: have a workspace created (or know how to create one).

Technical environment

• Ned2 robotic arm with firmware ≥ 5.4
• Custom gripper
• NiryoStudio version ≥ 1.3
• Connection Ned2 ↔ NiryoStudio via Hotspot, Wi-Fi, or Ethernet
• A Python IDE/editor (VS Code, PyCharm, etc.)
• A camera (webcam or robot’s camera)

Setup (safety & connections)

1. Place the Ned2 on a table with a clear obstacle-free perimeter of about 60 cm radius.
2. Plug the Ned2 into the safety box.
3. Connect the safety box to the Ned2’s power supply.
4. Connect the Ned2’s power supply to the electrical network.
5. Connect Ned2 to NiryoStudio and check communication.

Chapter 1 – Setup (10 min)

Step 1: Install required packages

Use your IDE’s integrated terminal or a system terminal:

pip install pyniryo2 easyocr opencv-python matplotlib keyboard

Step 2: Functions and methods used

Chapter 2 – Introduction to OCR & image-based control (10 min)

OCR (Optical Character Recognition) converts images containing text into machine-readable text.

1. Preprocessing: image enhancement (binarization, straightening, denoising).
2. Segmentation: splitting into text lines/zones.
3. Recognition: character identification (neural networks, etc.).
4. Post-processing: corrections using dictionaries/rules.

Workshop flow (first try)

1. Robot connection & calibration.
2. Show an image to the camera with text describing the piece (e.g., BLUE SQUARE).
3. Capture the image, detect text, extract color & shape, trigger vision_pick on the target workspace.
4. Move to the drop area and release the piece.

Example goal: pick a blue square if the text contains “BLUE” and “SQUARE”.

Chapter 3 – Your turn! (1h)

Goal: complete the program and cover multiple scenarios.

1. Currently, only the blue square is supported. Add the other pieces (shapes/colors). If the image only contains shape or color, use ObjectShape.ANY or ObjectColor.ANY.
2. Improve user feedback: messages like “Press Space to capture”, “Searching for piece…”, “No piece detected”, etc.

Suggestion: test with the robot’s camera instead of the webcam.

• Observation position of the workspace: make sure the angle is clear.
• Drop-off trajectories: add intermediate positions to avoid collisions.

Solution – Full code (with user feedback)

Replace ROBOT_IP and YOUR_WORKSPACE with your values. This code normalizes text to uppercase, chooses ANY by default if color/shape is missing, and ensures consistent PyNiryo2 API usage (consistent robot object).

Tips & troubleshooting

• EasyOCR is very slow: check that torch/torchvision are installed; if you have a compatible GPU, try gpu=True.
• Camera not detected: try cv2.VideoCapture(1), (2), etc., or close other apps using the camera.
• vision_pick fails: adjust lighting, orientation, workspace height, and check camera/robot calibration.
• keyboard on Linux: may require permissions for keyboard events (sudo).

Chapter 4 (extra) – Libraries used

OpenCV (cv2)

• cv2.imread(): read an image from a file
• cv2.imshow(): display an image in a window
• cv2.imwrite(): save an image
• cv2.GaussianBlur(): Gaussian blur

time

• time.sleep(secs): pause (used in this workshop)
• time.time(), time.ctime(), time.localtime(), time.gmtime()

matplotlib.pyplot

Display images in RGB via plt.imshow (useful since OpenCV works natively in BGR).

keyboard

• Capture global keyboard events, shortcuts, etc. (Windows, Linux with sudo, experimental support on macOS).

os

• System operations (paths, environment variables, etc.). Here: immediate exit via os._exit(0) (use with caution).

easyocr

Simple-to-use OCR. More advanced alternatives exist, but EasyOCR is ideal for a quick workshop.

💡 Don’t forget to adapt poses and speeds to your setup. Remember to version your code and document your settings (workspace, poses, lighting).