EMS-ESP-Boiler is an controller running on an ESP8266 to communicate with EMS (Energy Management System) based Boilers from the Bosch range. This includes the Buderus and Nefit ranger of boilers, heaters and thermostats.
There are 3 parts to this project, first the design of the circuit, second the code to deploy to an ESP8266 based microcontroller and lastly settings for Home Assistant to monitor data and issue direct commands via MQTT.
- EMS-ESP-Boiler
My original intention for this home project with to build my own smart thermostat for my Nefit Trendline boiler and then have it controlled via Home Assistant on my mobile phone. I had a few ESP32s and ESP8266s lying around from previous IoT projects and building a specific circuit to decode the EMS messages was a nice challenge into more complex electronic circuits. I then began adding new features such as timing how long the shower was running for and triggering an alarm (actually a shot of cold water!) after a certain duration.
Acknowledgments and kudos to the following people and their open-sourced projects:
susisstrolch - Probably the first working version of the EMS bridge circuit I could find designed for the ESP8266. I borrowed Juergen's schematic and parts of his code logic.
bbqkees - Kees built a circuit and wrote some sample Arduino code to read from the EMS and push messages to Domoticz. His SMD board is also available to purchase from him directly.
EMS Wiki - A comprehensive reference for decoding the EMS telegrams, which I found not always to be 100% accurate. It's in German so use Google Translate if you need help.
Most Bosch branded boilers that support the Logamatic EMS (and EMS+) bus protocols work with this design. Which are Nefit, Buderus, Worcester and Junkers and copyrighted.
I've tested the code and circuit with a few ESP8266 development boards such as the Wemos D1 Mini, Wemos D1 Mini Pro, Nodemcu0.9 and Nodemcu2 boards. It will also work on bare ESP8266 chips such as the 12s but do make sure you disabled the LED support and wire the UART correctly as this is switched (explained below).
- Either build the circuit below or purchase a ready built board from bbqkees via his GitHub page or the Domoticz forum.
- Get an ESP8266 dev board and connect the 2 EMS output lines from the boiler to the circuit and the Rx and Tx out to ESP pins D7 and D8 respectively. The EMS connection can either be the 12-15V AC direct from the thermostat bus line or from the 3.5" Service Jack at the front.
- Optionally connect the three LEDs to show Rx and Tx traffic and Error codes to pins D1, D2, D3 respectively. I use 220 Ohm pull-down resistors. These pins are configurable in
boiler.ino. This is further explained in the code section below. - Build and upload the firmware to the ESP8266 device. I used Platformio with Visual Studio. Do make sure you set the MQTT and WiFi credentials correctly and if you're not using MQTT leave the MQTT_IP blank. The firmware supports OTA too with the default hostname as 'boiler' (or 'boiler.' depending on your OS and how the mdns resolves hostnames).
- Power the ESP either via USB or direct into the 5v vin pin from an external power 5V volts supply with min 400mA.
- Attach the 3v3 out on the ESP8266 to the DC power line on the EMS circuit as indicated in the schematics.
- The WiFi connects via DHCP by default. Find the IP by from your router and then telnet (port 23) to it. Tip: to enable Telnet on Windows run
dism /online /Enable-Feature /FeatureName:TelnetClientor install something like putty. If everything is working you should see the messages appear in the window as shown in the next section. However if you're unable to locate the IP of the ESP then probably the WiFi failed to instantiate. In this case add -DUSE_SERIAL to the build options, connect at USB, build, upload and then use a terminal to connect to the serial port to see the debug messages. A word of warning, do not use both a USB and power from the EMS at the same time.
Use the telnet client to inform you of all activity and errors real-time. This is an example of the telnet output:
If you type 'v 3' and Enter, it will toggle verbose logging showing you more detailed messages. I use ANSI colors with white text for info messages, green for well formatted telegram packages (which have validated CRC checks), red for corrupt packages and yellow for send responses.
To see the current values of the Boiler and its parameters type 's' and hit Enter. Watch out for unsuccessful telegram packets in the #CrcErrors line.
Commands can be issued directly to the EMS bus typing in a letter followed by an optional parameter and pressing Enter. Supported commands are:
- r to send a read command to the boiler. The 2nd parameter is the type. For example 'b 33' will request type UBAParameterWW and bring back the Warm Water temperatures from the Boiler.
- t is similar, but to send a read command to the thermostat.
- T set the thermostat temperature to the given celsius value
- w to adjust the temperature of the warm water from the boiler
- a to turn the warm water on and off
- h to list all the recognized EMS types
- p to toggle the Polling response on/off (note it's not necessary to have Polling enabled to work)
- m to set the thermostat mode to manual or auto
- S to toggle the Shower Timer functionality on/off
- A to toggle the Shower Timer Alert functionality on/off
Disclaimer: be careful when sending values to the boiler. If in doubt you can always reset the boiler to its original factory settings by following the instructions in the user guide. On my Nefit Trendline HRC30 that is done by holding down the Home and Menu buttons simultaneously for a few seconds, selecting factory settings from the scroll menu and lastly pressing the Reset button.
The EMS circuit is really all credit to the hard work many people have done before me, noticeably susisstrolch with his ESP8266 version.
I've included a prototype boards you can build yourself on a breadboard. One part for only reading values from the Boiler and an extension with the write logic so you can also send commands.
We need the Rx/Tx of the ESP8266 for flashing, so the code in emsuart.cpp switches the UART pins to use RX1 and TX1 (GPIO13/D7 and GPIO15/D8 respectively). This also prevents any bogus stack data being sent to EMS bus when the ESP8266 decides to crash like after a Watch Dog Reset.
The breadboard layout was done using DIY Layout Creator and sources files are included in this repo.
The schematic used (as designed by susisstrolch):
Optionally I've also added 2 0.5A/72V polyfuses between the EMS and the two inductors L1 and L2 for extra protection.
And lastly if you don't fancy building the circuit, bbqkees can sell you one complete with SMD components which looks like the photo below when connected to a Wemos D1 Mini:
The EMS circuit will work with both 3.3V and 5V. It's easiest though to power directly from the ESP8266's 3V3 line and run a steady 5V into the microcontroller. Powering the ESP8266 microcontroller can be either
- via the USB if your dev board has one
- using an external 5V power supply into the 5V vin on the board
- powering from the 3.5" service jack on the boiler. This will give you 8V so you need a buck converter (like a Pololu D24C22F5) to step this down to 5V to provide enough power to the ESP8266 (250mA at least)
- powering from the EMS line, which is 15V A/C and using a buck converter as described above. Note the current design has stability issues when sending packages in this configuration so this is not recommended yet if you plan to many send commands to the thermostat or boiler.
| With Power Circuit |
|---|
 |
Packages are sent to the EMS "bus" from the Boiler and any other compatible connected devices via serial TTL transmission. The protocol is 9600 baud, 8N1 (8 bytes, no parity, 1 stop bit). Each package is terminated with a break signal <BRK>, a 11-bit long low signal of zeros.
A package can be a single byte (see Polling below) or a string of 6 or more bytes making up an actual data telegram. A telegram is always in the format:
[src] [dest] [type] [offset] [data] [crc] <BRK>
I reference the first 4 bytes as the header in this document.
Each device has a unique ID.
The Boiler has an ID of 0x08 (type MC10) and also referred to as the Bus Master or UBA.
My thermostat, which is a* Moduline 300* uses the RC30 protocol and has an ID 0x17. If you're using a RC35 type thermostat such as the newer Moduline 300s or 400s use 0x10 and make adjustments in the code as appropriate. bbqkees did a nice write-up on his github page here.
Our circuit acts as a service key and thus uses an ID 0x0B. This ID is reserved for special devices intended for installation engineers for maintenance work.
The bus master (boiler) sends out a poll request every second by sending out a sequential list of all possible IDs as a single byte followed by the break signal. The ID always has its high 7th bit set so in the code we're looking for 1 byte messages matching the format [dest|0x80] <BRK>.
Any connected device can respond to a Polling call with an acknowledging by sending back a single byte with its own ID. In our case we would listen for a [0x8B] <BRK> (meaning us) and then send back [0x0B] <BRK> to say we're alive and ready. Although I found this is not needed for normal operation so it's disabled as default in the code.
Polling is also the trigger to start transmitting any packages queued for sending. It must be done within 200ms or the bus master will time out.
When a device is broadcasting to everyone there is no specific destination needed. [dest] is always 0x00.
The tables below shows which types are broadcasted regularly by the boiler (ID 0x08) and thermostat (ID 0x17). The data length is excluding the 4 byte header and CRC and the Name references those in the ems wiki.
| Source (ID) | Type ID | Name | Description | Data length | Frequency |
|---|---|---|---|---|---|
| Boiler (0x08) | 0x34 | UBAMonitorWWMessage | warm water temperature | 19 bytes | 10 seconds |
| Boiler (0x08) | 0x18 | UBAMonitorFast | boiler temps, power, gas/pump switches | 25 bytes | 10 seconds |
| Boiler (0x08) | 0x19 | UBAMonitorSlow | boiler temp and timings | 22 bytes | 60 seconds |
| Boiler (0x08) | 0x1C | UBAWartungsmelding | maintenance messages | 27 bytes | 60 seconds |
| Boiler (0x08) | 0x2A | n/a | status, specific to boiler type | 21 bytes | 10 seconds |
| Boiler (0x08) | 0x07 | n/a | ? | 21 bytes | 30 seconds |
| Source (ID) | Type ID | Name | Description | Frequency |
|---|---|---|---|---|
| Thermostat (0x17) | 0x06 | RCTime | returns time and date on the thermostat | 60 seconds |
| Thermostat (0x17) | 0x91 | RC20StatusMessage | returns current and set temperatures | 60 seconds |
| Thermostat (0x17) | 0xA3 | RCTempMessage | returns temp values from external (outdoor) sensors | 60 seconds |
Refer to the code in ems.cpp for further explanation on how to parse these message types and also reference the EMS Wiki.
Telegram packets can only be sent after the Boiler sends a poll to the sending device. The response can be a read command to request data or a write command to send data. At the end of the transmission a poll response is sent from the client (<ID> <BRK>) to say we're all done and free up the bus for other clients.
When doing a request to read data the [src] is our device (0x0B) and the [dest] has it's 7-bit set. Say we were requesting data from the thermostat we would use [dest] = 0x97 since RC30 has an ID of 0x17.
When doing a write request, the 7th bit is masked in the [dest]. After this write request the destination device will send either a single byte 0x01 for success or 0x04 for fail.
Every telegram sent is echo'd back to Rx.
Disclaimer: This code here is really for reference only, I don't expect anyone to use as is since it's highly tailored to my environment and my needs. Most of the code however is self explanatory with comments here and there. If you wish to make some changes start with the defines and const sections at the top of boiler.ino.
The code is built on the Arduino framework and is dependent on these external libraries:
- Time http://playground.arduino.cc/code/time
- PubSubClient http://pubsubclient.knolleary.net
- ArduinoJson https://github.com/bblanchon/ArduinoJson
- CRC32 https://github.com/bakercp/CRC32
emsuart.cpp handles the low level UART read and write logic. You shouldn't need to touch this. All receive commands from the EMS bus are handled asynchronously using a circular buffer via an interrupt. A separate function processes the buffer and extracts the telegrams. Since we don't send too many write commands this is done sequentially. I couldn't use the standard Arduino Serial implementation because of the 11-bit break signal causes a frame-error which gets ignored.
ems.cpp is the logic to read the EMS packets (telegrams), validates them and process them based on the type.
boiler.ino is the Arduino code for the ESP8266 that kicks it all off. This is where we have specific logic such as the code to monitor and alert on the Shower timer and light up the LEDs. LED support is enabled by setting the -DUSE_LED build flag.
ESPHelper.cpp is my customized version of ESPHelper with added Telnet support and some other minor tweaking.
ems.cpp defines callback functions that handle all the broadcast types listed above (e.g. 0x34, 0x18, 0x19 etc) plus these extra types:
| Source (ID) | Type ID | Name | Description |
|---|---|---|---|
| Boiler (0x08) | 0x33 | UBAParameterWW | reads selected & desired warm water temp |
| Boiler (0x08) | 0x14 | UBATotalUptimeMessage | |
| Boiler (0x08) | 0x15 | UBAMaintenanceSettingsMessage | |
| Boiler (0x08) | 0x16 | UBAParametersMessage | |
| Thermostat (0x17) | 0xA8 | RC20Temperature | sets operating modes for a RC20 & RC30 |
| Thermostat (0x17) | 0x02 | Version | reads Version major/minor |
| Thermostat (0x18) | 0x0A | EasyTemperature | thermostat monitor for an TC100/Easy |
In boiler.ino you can make calls to automatically send these read commands. See the function regularUpdates()
The code is originally designed for a Moduline300 (RC30) thermostat.
To adjust for a RC35 first change EMS_ID_THERMOSTAT in ems.cpp. A RC35 thermostat has 4 heating circuits and to read the values use different Monitor type IDs (e.g. 0x3E, 0x48, etc). The mode (0=night, 1=day, 2=holiday) is the first byte of the telegram and the temperature is the value of the 2nd byte divided by 2. Then to set temperature values use the Working Mode with type IDs (0x3D, 0x47,0x51 and 0x5B) respectively. Set the offset (byte 4 of the header) to determine which temperature you're changing; 1 for night, 2 for day and 3 for holiday. The data value is the desired temperature multiplied by 2 as a single byte.
There is limited support for an Nefit Easy TC100) thermostat. The current room temperature and setpoint temperature can be read. What still needs fixing is
- reading the mode (manual vs clock)
- setting the temperature
- To configure for your thermostat and specific boiler settings, modify
my_config.h. Here you can- set the thermostat type. The default ID is 0x17 for an RC30 Moduline 300.
- set flags for enabled/disabling functionality such as
BOILER_THERMOSTAT_ENABLED,BOILER_SHOWER_ENABLEDandBOILER_SHOWER_TIMER. - Set WIFI and MQTT settings, instead of doing this in
platformio.ini
- To add new handlers for EMS data types, first create a callback function and add to the
EMS_Typesarray at the top of the fileems.cppand modifyems.h
When the ESP8266 boots it will send a start signal via MQTT. This is picked up by Home Assistant and sends a notification informing me that the device has booted. Useful for knowing when the ESP gets reset.
I'm using the standard PubSubClient client so make sure you set -DMQTT_MAX_PACKET_SIZE=400 as the default package size is 128 and our JSON messages are around 300 bytes.
I run Mosquitto on my Raspberry PI 3 as the MQTT broker.
The boiler data is collected and sent as a single JSON object to MQTT TOPIC home/boiler/boiler_data (or {hostname}/boiler_data for ESPurna). A hash is generated (CRC32 based) to determine if the payload has changed, otherwise don't send it. An example payload looks like:
{"wWCurTmp":"43.0","wWHeat":"on","curFlowTemp":"51.7","retTemp":"48.0","burnGas":"off","heatPmp":"off","fanWork":"off","ignWork":"off","wWCirc":"off","selBurnPow":"0","curBurnPow":"0","sysPress":"1.6","boilTemp":"54.7","pumpMod":"4"}
Similarly the thermostat values are sent as a json package under a topic named home/boiler/thermostat_data with the current mode, room temperature and set temperature.
These topics can be configured in the TOPIC_* defines in boiler.ino. Make sure you change the HA configuration too to match.
Checking whether the shower is running was tricky. We know when the warm water is on and being heated but need to distinguish between the central heating, shower, hot tap and bath. I found via trial and error the Selected Burner Max Power is between 80% and 115% when the shower is running and fixed at 75% if the central heating is on. Furthermore the Selected Flow Impulsion is 80 C for the heating.
There is other logic in the code to compensate for ramp up and whether the shower is turned off and back on again quickly within a 10 second window.
Within Home Assistant it renders as:
and the alerts on an iOS/Android device using PushBullet, PushOver or any notification service would look like:
You can find the .yaml configuration files under doc/ha. See also https://community.home-assistant.io/t/thermostat-and-boiler-controller-for-ems-based-boilers-nefit-buderus-bosch-using-esp/53382
PlatformIO is my preferred way. The code uses a modified version ESPHelper which handles all the basic handling of the WiFi, MQTT, OTA and Telnet server. I switched from Atom to the marvelous Visual Studio Code, works on Windows, OSX and Linux.
On Windows:
- Download Git (install using the default settings)
- Download and install Visual Studio Code (VSC)
- Restart the PC (if using Windows) to apply the new PATH settings. It should now detect Git
- Install these VSC extensions: PlatformIO IDE & GitLens, and then click reload to activate them
- Git clone this repo, eith using
git clonefrom PlatformIO's terminal or the Git GUI interface - Create a
platformio.inibased on theplatformio.ini-examplemaking the necessary changes for your WiFi and MQTT credentials in the build flags. If you're not using MQTT leave MQTT_IP empty (MQTT_IP="")
On Linux (e.g. Ubuntu under Windows10):
- make sure Python 2.7 is installed
% pip install -U platformio
% sudo platformio upgrade
% platformio platform update
% git clone https://github.com/proddy/EMS-ESP-Boiler.git
% cd EMS-ESP-Boiler
% cp platformio.ini-example platformio.ini- edit
platformio.inito setenv_defaultand the flagsWIFI_SSID WIFI_PASSWORD, MQTT_IP, MQTT_USER, MQTT_PASS. If you're not using MQTT leave MQTT_IP empty (MQTT_IP="")
% platformio run -t uploadESPurna is framework that handles most of the tedious tasks of building IoT devices so you can focus on the functionality you need. This replaces my ESPHelper code in the standalone version above. ESPurna is natively built on PlatformIO and Visual Studio Code too which is nice. The latest version I tested this on is 1.13.3. So if you're brave, follow these steps:
- Download and install NodeJS. This gives you npm. Choose the LTS version
- Download ESPurna by cloning the ESPurna git repository (command palette, git clone, https://github.com/xoseperez/espurna.git)
- Restart VSC.
- From VSC open the folder
espurna\code. PlatformIO should detect and set some things up for you automagically - open a terminal window (ctrl-`)
- Install the node modules:
npm install --only=dev - Build the web interface:
node node_modules/gulp/bin/gulp.js. This will create a compressedcode/espurna/static/index.html.gz.h. If you get warnings about lf during the building editgulpfile.jsand change the line'failOnError': truetofalseas a temporary workaround. - Modify the platformio.ini file making sure you add
-DUSE_CUSTOM_H -DUSE_EXTRAto thedebug_flags - Copy the following files from EMS-ESP-Boiler repo to where you installed ESPurna
espurna/index.html -> code/html/index.html
espurna/custom.h -> code/config/custom.h
espurna/boiler-espurna.ino -> code/espurna/boiler-espurna.ino
ems*.* -> code/espurna/- Now build and upload as you usually would with PlatformIO (or ctrl-arl-t and choose the right build). Look at my version of platformio.ini as an example.
- When the firmware loads, use a wifi connected pc/mobile to connect to the Access Point called ESPURNA_XXXXXX. Use 'fibonacci' as the password. Navigate to
http://192.168.4.1from a browser, set a new username and password when prompted, log off the wifi and reconnect to the AP using these new credentials. Again go to 192.168.4.1 - In the ADMIN page enable Telnet and SAVE
- In the WIFI page add your home wifi details, click SAVE and reboot, and go to the new IP
- Configure MQTT
The Telnet functions are BOILER.READ, BOILER.INFO and a few others for reference. HELP will list them. Add your own functions to expand the functionality by calling the EMS* functions as in the examples.
If you run into issues refer to ESPurna's official setup instructions here and here.
This is what ESPurna looks like with the custom boiler code:
Note: I didn't bother porting all the EMS Read and Write commands from the Telnet code to the Espurna, but its pretty straight forward if you want to extend the code.
pre-baked firmwares for some ESP8266 devices based on ESPurna are available in the directory /firmware which you can upload yourself using esptool bootloader. On Windows, follow these instructions:
- Check if you have python 2.7 installed. If not download it and make sure you select the option to add Python to the windows PATH
- Install the ESPTool by running
pip install esptoolfrom a command prompt - Connect the ESP via USB, figure out the COM port
- run
esptool.py -p <com> write_flash 0x00000 <firmware>where firmware is the.binfile and <com> is the COM port, e.g.COM3
now follow the steps in ESPurna section above from #10 on to configure the device.
Porting to the Arduino IDE can be a little tricky but it is possible.
- Add the ESP8266 boards (from Preferences add Additional Board URL
http://arduino.esp8266.com/stable/package_esp8266com_index.json) - Go to Boards Manager and install ESP8266 2.4.x platform
- Select your ESP8266 from Tools->Boards and the correct port with Tools->Port
- From the Library Manager install ArduinoJson 5.13.x, PubSubClient 2.6.x, CRC32 and Time
- The Arduino IDE doesn't have a common way to set build flags (ugh!) so you'll need to un-comment these lines in
boiler.ino:
#define WIFI_SSID "<my_ssid>"
#define WIFI_PASSWORD "<my_password>"
#define MQTT_IP "<broker_ip>"
#define MQTT_USER "<broker_username>"
#define MQTT_PASS "<broker_password>"- Put all the files in a single sketch folder (
ESPHelper.*, boiler.ino, ems.*, emsuart.*) - cross your fingers and hit CTRL-R to compile...
Some annoying issues that need fixing:
- Very infrequently an EMS write command is not sent, probably due to a collision somewhere in the UART between an incoming Rx and a Poll. The retries in the code fix this for now.
- I've seen a few duplicate telegrams being processed. Again, it's harmless and not a big issue, but still a bug.
Here's my top things I'm still working on:
- Complete the ESP32 version. It's surprisingly a lot easier doing the UART handling on an ESP32 with the ESP-IDF SDK. I have a first version beta that is working.
- Find a better way to control the 3-way valve to switch the warm water off quickly rather than deactivating the warm water heater each time. There is an unsupported call to do this, but I think its too risky and experimental. I'd love to get my hands on an Nefit Easy, sniff the packets being sent and reverse engineer the logic. Anyone help?
Any comments, suggestions or code changes are very welcome. Please post a GitHub issue.