Your first LLM thing
Wire a microcontroller kit to a language model, so a physical object can be given a character and answer for itself.
Requirements
- Connected Interaction Kit
- OpenAI API key
This workshop session is based on a research-through-design project called Prompting Realities. Run by Mahan Mehrvarz for the Speculative Design Studio course, MSc Design for Interaction, TU Delft Industrial Design Engineering, 2025.
Before you start
You need your own MQTT broker before you start
MQTT is a small publish/subscribe protocol for machine-to-machine messages: a device publishes a short message to a named topic, a broker passes it on, and anything subscribed to that topic receives it. Everything here goes through one broker.
The broker used on the day belonged to the faculty. Its dashboard is still visible in the screenshots, but its address and credentials are not published and you will not be able to connect to it. Set up your own: sign up at shiftr.io and create a broker — its page gives you the host, a username and a password. Those are the three values written below as <your-namespace>.cloud.shiftr.io, <your broker username> and <your broker password>. Anything else written as <like this> is likewise yours to fill in, including the Wi-Fi the device joins.
Any MQTT broker will do — shiftr.io is used here because its live dashboard draws the traffic as it happens, which makes debugging a silent device much easier. Its documentation covers the rest.

Connecting the boards to the internet
You can find a lot of useful code and information about your kit on their website. We are following the preparation tutorial from the original Connected Interaction Kit.
First let's prepare the setup
Connect your board via the USB cable to your computer.

Installing the proper interface to write and edit code on your board
Before you write your first program, you need to complete one last step:
- Visit codewith.mu and click the green Download button.
- Select the correct link for your operating system (Windows or Mac OS).
- Once the download is complete, launch the installer and follow the instructions on the screen.
- On its first run, Mu will ask you to select a mode. Choose CircuitPython.

After you are done, you should see the Mu editor up and running on your laptop.

Quick test
Connect the chainable NeoPixel LED to the pin (D13) of your board.

Make sure you connect the grove cable to the in side.
For more information about the RGB LED, see this link.
Open the code.py file and replace its content with the code below. Don't forget to save.
# --- Imports
import time
import json
import board
import neopixel
from MQTT import Create_MQTT
from settings import settings
# --- Variables
# Configure the pin here (any NeoPixel-capable pin)
LED_PIN = board.D13
NUM_LEDS = 1
led = neopixel.NeoPixel(
LED_PIN,
NUM_LEDS,
auto_write=False,
pixel_order=neopixel.GRBW,
)
# Current LED state: "[R, G, B, Brightness]"
led_state = [255, 255, 255, 200]
# MQTT setup
client_id = settings["mqtt_clientid"]
mqtt_topic = settings.get("mqtt_topic")
mqtt_client = Create_MQTT(client_id)
# --- Functions
def apply_single_led(led_object, values):
"""Apply [R, G, B, Brightness] to a single NeoPixel object."""
if not isinstance(values, list) or len(values) != 4:
return
r, g, b, brightness = values
# Brightness 0-255 -> 0.0-1.0
led_object.brightness = max(0.0, min(1.0, brightness / 255.0))
# With pixel_order=neopixel.GRBW, we still pass (R, G, B, W)
led_object.fill((r, g, b, 0))
led_object.show()
def apply_led():
"""Apply current state to the single LED."""
apply_single_led(led, led_state)
def on_message(client, topic, message):
"""Handle incoming MQTT messages to update the LED color."""
global led_state
try:
data = json.loads(message)
# Expect messages like: {"led": [R, G, B, Brightness]}
if "led" in data and isinstance(data["led"], list) and len(data["led"]) == 4:
led_state = data["led"]
apply_led()
print("Updated led:", led_state)
else:
print("Received invalid LED payload:", data)
except Exception as e:
print("Error parsing MQTT message:", e)
# --- Setup
# Clear LED first
led.fill((0, 0, 0, 0))
led.show()
# Apply initial state
apply_led()
# Configure MQTT callbacks and subscription
mqtt_client.on_message = on_message
mqtt_client.subscribe(mqtt_topic)
print("Subscribed to topic:", mqtt_topic)
# --- Main loop
while True:
mqtt_client.loop(timeout=0.2)
time.sleep(0.1)Connect your board to the laptop and load the settings.py file. Replace <Your device name> with your actual group name and make sure the rest looks like below. You don't need the < > signs there!
settings ={
"ssid": "<your Wi-Fi network>",
"password": "<your Wi-Fi password>",
"mqtt_clientid": "<Your device name>",
"broker": "<your-namespace>.cloud.shiftr.io",
"mqtt_user": "<your broker username>",
"mqtt_password": "<your broker password>",
"mqtt_port": 1883,
"mqtt_topic": "RGBLight"
}You should see this in the serial monitor of the Mu editor:

Click here:

Follow up with the instructions in the class.
Getting up and running with promptingrealities.com

You need to sign up and make an account. A magic link is sent to your email address and by clicking on the link (also check your spam folder) you'll be logged in.

From the left side panel, add a new LLM thing.
A new LLM thing is added and now you can configure it based on the first example. The first step is to give it a name. In this example we call it Neopixel.

The main part of the interface is the configuration studio. In order to create an LLM thing you need to add information to the four tabs:
- Prompt Instruction
- JSON Schema
- MQTT
- OpenAI API Key
A quick way would be to use one of the examples from the Prompting Realities GitHub repository. Let's do one single RGB LED.
Creating your first LLM thing
Use the text below in your Prompt Instruction text field.
You are sending values for a RGB LED to change its color based on the command or prompt from the user.
You send three values with each response.
R: for red
G: for green
B: for blue
X: the last value is the brightness
-Here is an example of the values inside the values object
{
"led": [255, 100, 50, 200]
}
Ds:
-You cannot really do beyond this color change. If the user asked for other things be transparent.
-If user says something not about the LED. You can answer but don't change the color.
Do Not Do:
-in the response object do not include any technical term about the RGB values. Just refer to colors you created and engage the users in conversation.
-Don't be wordy keep it very short.Use the JSON Schema below.
{
"name": "led",
"schema": {
"type": "object",
"required": [
"answer",
"MQTT_value"
],
"properties": {
"answer": {
"type": "string"
},
"MQTT_value": {
"type": "object",
"required": [
"led"
],
"properties": {
"led": {
"type": "array",
"items": {
"type": "number"
},
"maxItems": 4,
"minItems": 4
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
"strict": true
}The JSON schema defines exactly one acceptable shape of data for something called led. Its job is to:
- say what fields must exist
- say what type each field must be
- forbid anything extra
- enforce exact sizes and structures
This schema enforces the following:
- a human-readable message (
answer) - a machine-readable LED command (
MQTT_value.led) - the LED command is always four numbers
- nothing else is tolerated
In the MQTT tab you have the following fields to fill in: Broker Host address, Username, port, password, Topic.

Broker, Username, Port and Password are often the same as in your settings.py file (check Part B, step 3).
broker: <your-namespace>.cloud.shiftr.io
username: <your broker username>
port: 1883
password: <your broker password>But what about the topic? The topic is something shared between your LLM thing and your hardware (CircuitPython board). Both should subscribe to the same topic. This prevents your messages from being sent to other boards and is key for communication between the LLM thing and your board. Just pick something simple — myled, mahan-led, light1, coloredlight.
The last step is to add a valid OpenAI API key. You will be given one if you are part of a workshop, but in general any OpenAI API key works.
Press save!
Now, as you see, your Run LLM thing button in the top right corner is green and you can go ahead and run it.

Below the configuration panel you'll see the client access panel. You can open the chat interface, copy the link and paste it somewhere, or scan the QR code to go straight to the chat interface.

You can start to chat with your RGB LED now. Enjoy!

Exploring other actuators in the Kit
You can explore other examples from the GitHub folder. You can chat with a servo, a buzzer or a vibration motor from your kit. The preferred order is: two LEDs, vibration motor, servo motor, buzzer.