Posts on Random Musings

MXroute Email Aliases - A SimpleLogin Replacement

Mon, 30 Mar 2026 00:00:00 +0000

Recently I discovered that my favorite email service (MXroute) introduced an API which has opened up a lot of fun new possibilities with the service, including letting me recreate most (if not all) of the functionality of SimpleLogin. In this post I’m going to detail some of what SimpleLogin is, why you should use a service like it, and if you’re an MXroute user some cool apps to check out.

SimpleLogin

So what is SimpleLogin? Simply (heh) put, it’s a service that lets you create email forwarders. For example, if you’re signing up for some sketchy service you aren’t 100% sure about, you can create sketchyservice@example.com as an email address and sign up with that. All the email sent to that address can then be forwarded to your actual email address and nobody on the other side is the wiser.

I’ve been using SimpleLogin now since 2021 (so almost 5 years) and really have had few complaints. It got bought/acquired by Proton in 2022 (the people behind ProtonMail and other privacy-focused services) and IMO I think Proton has done a good job of keeping the product alive, though I haven’t seen a whole lot of innovation/development.

As mentioned above, the main usecase is to sign up for services with what is essentially a disposable email address. Instead of having to worry about not being able to unsubscribe to emails from a company (I think we all know one company/email that we just cannot unsubscribe to), you just turn off/delete the forwarder and no more emails! The other advantage I see is that if there is a data breach, your real email address isn’t out there. Moreover, having forwarder addresses like this lets you track down the source of a breach (or even see if a company sold your email address/data to another company).

I tried self-hosting SimpleLogin on a number of occasions because it is open source, but I could never get it to work. Because self-hosting email is awful. And nobody should do it unless you truly enjoy bashing your head against the desk and fighting spam/IP block lists. Given the relatively low price of SimpleLogin ($30 for the year, though looks like they raised the price to $36 a year) I was content to continue paying for the service.

I did have a slight annoyance with SimpleLogin (that is not their fault): I want to actually use my root domain name for normal email, so I had to put SimpleLogin on a subdomain for email. So my main email is @example.com and SimpleLogin is hosted at @sl.example.com. Not the worst thing but it is ever so slightly annoying if I have to give out the email address and potentially explain that yes, there is a period in the domain name part.

MXroute

Since self-hosting emails is not an option, a few years ago I started exploring alternatives to GMail (because Google being Google). First I tried ProtonMail, but didn’t like the Android app (didn’t sync read emails for notifications, which drove me crazy) and having to use the Bridge to connect to something like Thunderbird wasn’t the end of the world but was limiting. Then I tried FastMail and generally liked that as an option. But then I stumbled onto MXroute in early 2022 and I have not looked back. The first year I purchased a yearly subscription, but on Black Friday that year for just $100 I got a lifetime subscription to 10 gigs of storage. The packages have changed and prices have gone up (and there have been changes to the lifetime plan), but I still think it’s worth it.

This is in no ways a paid advertisement, but I simply cannot rave enough about MXroute. The owner, Jarland, has quite the personality if you browse around various forums and places you’ll see what I mean. In my very few interactions with him via support tickets, the guy clearly knows his stuff and is incredibly passionate about what he does. Moreover, he takes things seriously. If you at all end up on his bad side (cough cough spammer cough cough) he does not pull his punches.

Also, it’s important to note that there are no free trials. My advice if you’re interested in the service is buy a year, test it out, then go for the Black Friday sale. Usually there are some good deals.

MXroute Aliases

So there I was one day a week ago, logging into the webmail page for my email and I noticed I had an SSL cert error. Well that was weird. So I did some poking around trying to figure out what was going on, which led me to the MXroute subreddit.* And that’s where I learned that not only does MXroute offer unlimited email aliases/forwarding, but there’s a new API (as of 3 months ago) for setting up those aliases. And people have taken advantage and written some really cool apps!

The first I found is mxroute-alias-manager. This is what sold me on ditching SimpleLogin. It’s incredibly easy to set up and has a really straightforward and polished UI for creating and managing your email aliases on MXroute. Really there’s not much to the setup: it’s just one container, you need to generate an API key from MXroute and configure your email server/username/email domains to use. And boom, all set.

Oh…I did have some permission issues with the docker container in bind mount. So you may need to create (I used touch) the following files: config.json, dest_templates.json, labels.json, tags-metadata.json, and users.json

The second app I’ve run into and tested so far is a Bitwarden integration. Bitwarden offers options to generate usernames/email addresses with services like SimpleLogin or AnnonAddy, which simplifies the creation of mostly anonymous logins. With the MXroute Bitwarden integration, you can generate aliases on MXroute for Bitwarden. The only “gotcha” I had with this one is that I needed to set up a reverse-proxy with SSL in order to use it with the Android app. It did not like using a non-HTTPS IP address of the local server. Which is not an issue for me, so more of a heads up.

Conclusion

In the end, I spent an evening migrating all my aliases off of SimpeLogin and moved my email completely to MXroute. I wasn’t able to migrate some logins off of my sl.example.com setup, so unfortunately I’ve had to retain that for legacy purposes. But now if I ever have to provide an email address over the phone or in person it’ll be a little bit easier to say. I can now save $36 a year, which really wasn’t my main issue with SimpleLogin but I’ll happily save on one more subscription.

In summary: if you’re using MXroute, take a look at the awesome projects out there! It’s really incredible and I think it’s just the beginning.

I did ultimately figure out what my SSL cert issue was. Turns out there’s a new management panel that largely supplants the old DirectAdmin control panel, and somehow some of my settings involving the SSL certs got wonky. That is probably my only complaint about MXroute: it’s not the easiest service to get configured and things move around, but I do think things are getting better and the new panel is a step in the right direction. Once I reconfigured the SSL certs in the new panel, everything updated.

Printing HomeBox QR Code Labels to HomeAssistant over MQTT

Tue, 30 Dec 2025 00:00:00 +0000

Yes, it’s the end of another year. And while I could look back and reflect on the year, I’m instead looking forward to next year and some things I want to accomplish. I’m not going to necessarily call them “resolutions” as I kinda hate that idea, but I would like to keep better track of all the stuff in my house. Things like how many of a specific smart switch I have, do I have any spare SSDs, when did I buy that TV for warranty purposes. All that info would be helpful to have in any easily accessible place.

Update 1/10/2026: Reddit user /u/slurdge made some improvements to the QR code output. You can read more here. Their tweaks to the labels have been incorporated below. You can also follow their guide to add MQTT to the container and avoid the need for scripts.

Homebox Setup

I will admit that I have not spent much time setting up HomeBox yet, but in short it’s software to help you take inventory of and organize things in your house. It’s pretty feature rich, with lots of customization and data that can be stored. Essentially it does exactly what it says on the box. It’s something I’ve known about but never really played around with.

I give full credit to /u/flybrys on Reddit for the inspiration here (and really the inspiration to finally take a look at HomeBox) as they did most of the work figuring this out. Also, for pointing out the Niimbot D110, which is just $18 on Amazon as a label printer. It works over Bluetooth, so you’ll need some Bluetooth proxies to use it with HomeAssistant. I already have some ESP32’s set up to do just that so this was not an issue for me.

In terms of the setup from /u/flybrys, I’ve mostly just made some tweaks to streamline things. Namely their setup used network shares (and a number of scripts) and I am instead using MQTT. There’s nothing wrong with network shares, but in my homelab the LXC that runs HomeAssistant is 1) on a different VLAN and 2) doesn’t have network shares enabled. I could have resolved both of those, but I really didn’t want my IOT stuff having access to a network mount, so I endeavored to find another solution.

To start, here is my Docker Compose snippet to setup HomeBox:

  homebox:
    image: ghcr.io/sysadminsmedia/homebox:0.22.3
    container_name: homebox
    restart: unless-stopped
    environment:
      - HBOX_LOG_LEVEL=info
      - HBOX_LOG_FORMAT=text
      - HBOX_WEB_MAX_UPLOAD_SIZE=10
      - HBOX_LABEL_MAKER_PRINT_COMMAND=cp {{.FileName}} /labels/label.png
      - HBOX_LABEL_MAKER_PADDING=5
      - HBOX_LABEL_MAKER_MARGIN=5
      - HBOX_LABEL_MAKER_WIDTH=320
      - HBOX_LABEL_MAKER_HEIGHT=96
      - HBOX_LABEL_MAKER_FONT_SIZE=24
      - HBOX_OPTIONS_ALLOW_ANALYTICS=false
    volumes:
      - "{{ appdata_path }}/homebox/data:/data/"
      - "{{ appdata_path }}/homebox/labels:/labels/"
    ports:
      - 7745:7745

Key things to note:

Make sure you update the image tag to whatever the latest version is!
The HBOX_LABEL_MAKER_PRINT_COMMAND is necessary to enable the “Print on Server” button. All we need is a simple cp command to copy the QR code label out of the container to the mounted directory /labels. {{.FileName}} is straight from the HomeBox docs.
The rest of the padding/width/height specs come straight from /u/flybrys’s post.

To HomeAssistant!

Once you spin up HomeBox and create a user/your first item, you’ll now have the option to “Print from Browser”. If you click it you’ll see a “Print on Server” button. Right now if you click it, all it will do is create a label.png file in the directory you mounted in /labels. Let’s fix that so it does more now.

The next step is to create a bash script that uses inotify to monitor the /labels directory for a new PNG file. When it detects a new PNG file in the directory, it fires off the image to MQTT. We’ll use this to get the image into HomeAssistant pretty seamlessly.

Sidenote: The reason for the script outside the container is that the HomeBox image does not contain mosquitto-client, so we have to go external in order to push it over MQTT. Perhaps someday mosquitto will be built into the container and you won’t need this part.

#!/bin/bash

WATCH_DIR="PATH-TO-HOMEBOX-LABELS"
MQTT_BROKER="BROKER-IP-ADDRESS"
MQTT_USERNAME="MQTT-USERNAME"
MQTT_PASSWORD="MQTT-PASSWORD"
MQTT_TOPIC="homebox/labels"

inotifywait -m -e close_write,moved_to "$WATCH_DIR" --format '%w%f' |
while read FILE
do
    if [[ "$FILE" == *.png ]]; then
        echo "Detected new PNG: $FILE"
        # Publish file via MQTT
        mosquitto_pub -h "$MQTT_BROKER" -u "$MQTT_USERNAME" -P "$MQTT_PASSWORD" -t "$MQTT_TOPIC" --retain -f "$FILE"
    fi
done

You can remove the MQTT username and password fields/arguments if you aren’t using them for your MQTT setup. The rest should all be customized as applicable. You can test it by running the script (mine is called monitor_homebox.sh, so ./monitor_homebox.sh worked for me). Note you will probably want to run the script as root since the label.png file will be owned by root with limited permissions. You could tweak the HBOX_LABEL_MAKER_PRINT_COMMAND environment variable to run a bash script like /u/flybrys does and chmod the file. I didn’t feel like taking that step, so I just run it as root. /shrug.

Go back to HomeBox and click the “Print on Server” button and check your MQTT server to see if there’s a new entry. Now to run the script all the time in order to monitor the folder for any new labels created, I just created a simple systemd service file, like the one below (again notice I’m running this as root, you can change that if you are uncomfortable with the approach).

[Unit]
Description=Homebox Label Monitor Service
After=network.target

[Service]
ExecStart=PATH/TO/YOUR/SCRIPT/monitor_homebox.sh
Restart=always
User=root

[Install]
WantedBy=multi-user.target

Don’t forget to enable and start the systemd service (and check to make sure it’s running) before proceeding to the next step.

Final Steps

Almost there! To tie the Niimbot D110 into HomeAssistant, you’ll need this integration. Install it (HACS is easiest as a custom repository). You’ll also want to set up an MQTT Camera entity in your config based on the MQTT topic you set up in the monitor_homebox.sh script. Mine is just this:

mqtt:
  - camera:
    - name: Homebox Labels
      topic: homebox/labels

And reboot HomeAssistant. Now add your D110 to HomeAssistant as a device (may have to scroll depending on how many Bluetooth devices you have in range).

I don’t do a lot of HomeAssistant automations (I use NodeRed) but I’m fairly certain there’s an MQTT trigger, so you can trigger the below action based on that trigger.

action: niimbot.print
data:
  payload:
    - type: dlimg
      url: >-
        http://HA_IP:HA_PORT/{{ state_attr('camera.YOUR_CAMERA_NAME',
        'entity_picture') }}
      x: 0
      "y": 0
      xsize: 320
      ysize: 96
  width: 320
  height: 96
  rotate: 90
  print_line_batch_size: 10
  wait_between_print_lines: 0.01
  density: 5
  preview: false
target:
  device_id: YOUR_D110_DEVICE_ID

A few things.

You can play with print_line_batch_size. /u/flybrys used 32. From my own trial and error I’ve settled on 10. My understanding is it controls print speed, and I found 10 to be fine.
The Niimbot requires a URL in order to print and can’t print a local file. Hence why we set up an MQTT camera. Using the template pulls the necessary camera API token to allow printing.
You can change preview to true if you want to test without physically printing anything. You can see what would have been printed if you look at the Last Label Made entity.
To find the device_id, go to the Actions screen and use the UI to select the Niimbot device and then switch back to YAML mode.

That should be it! Now if you go into HomeBox and click the “Print on Server” button a label should be printed within a few seconds on your Niimbot. Pretty sweet!

Happy organizing!

Migrating to PostgreSQL

Fri, 26 Dec 2025 00:00:00 +0000

I’ve been taking some time off work to enjoy the holidays, and decided that now was a good time to take on a few interesting projects I’ve wanted to do in the good ole homelab before the end of the year. The main one I accomplished today was migrating from MariaDB to PostgreSQL for the majority of my database backends.

I’ve long been a user of MariaDB for all my database needs. It has done the job and upgrades have been relatively straightforward, even between major versions. And there has been nothing really “wrong” with my setup. But over the years I’ve seen a lot more positive opinions express over PostgreSQL and a lot more projects supporting it (sometimes as the only option). I’ve run it for those few instances where I’ve needed it, but I was never a huge fan because to upgrade between major versions (ie going from 14–>15) in Docker required you to basically dump your entire database, create a new container, and restore the backup. That being said, PostgreSQL supports major versions for a long time, so I never feel compelled to upgrade that often.

What really prompted me to start looking into migrating was the deprecation of the MariaDB backend for Umami, the software I use to track visits to this blog. As much as I like Umami and the relative simplicity to set it up, I’ve been bitten twice by bad migrations involving the database, forcing me to start fresh. Going from MariaDB to PostgreSQL was the second instance. However, I’m hopeful that this will be the last time as I really don’t want to have to deal with it again. And the developer has indicated that PostgreSQL is easier to support for the software so….fingers crossed.

It just so happened at the time I was doing this conversion (over Thanksgiving when I was visiting family and bored), TheOrangeOne shared an interesting container that removes the major pain point I’ve had with PostgreSQL: automatic upgrades. The container is pgautoupgrade/pgautoupgrade, and just like the name implies, it handles the upgrades for you. Obviously you should still backup before upgrading, but this container allowed me to upgrade my existing PostgreSQL instances without any issues to the latest version. You can read more about the container here.

As a result of the Umami database conversion, the only other DB I had left on my VPS is for Gitea, so I figured I would look into migrating my MariaDB backend for Gitea to PostgreSQL. And oooof, it was not easy. I played around with various sql dumps, pgloader, and everything in between. After faffing about for several hours, I was ultimately successful in migrating the data over (though don’t ask me how now). Huzah, two fewer instances of MariaDB and only one image/database type on my VPS.

After the pain of the Gitea migration I took a bit of a break. But I got in a homelabbing mood today setting up Booklore so figured I would take the rest of the migrations I could. And overall I was successful in migrating the following apps from MariaDB to PostgreSQL. I’ve sorted them by ease of migration:

Nextcloud - Super simple. There’s an occ-based migration tool, occ db:convert. You can read more about it here. I spun up a new PostgreSQL container, filled out the command and ran it. Took about 5 minutes and everything was migrated over. It even handled updating Nextcloud’s config.php file.
Vaultwarden - Relatively simple. Followed the instructions in the Github here. Basically spun up a new Vaultwarden instance (with the new PostgreSQL database), stopped both the old and new instances, and used pgloader to migrate the data. Only complication for me was that my database was old so it was bitwarden_rs and not vaultwarden in the load file’s ALTER SCHEMA line.
HomeAssistant - On par with the Gitea migration, but for a different reason. My HA database is quite large, with a few million entries, which is not surprising given how many entities I have. I therefore worked with my AI buddy Claude to tweak the settings for pgloader to ultimately get it to work. The main struggle was with memory limits in both the databases as well as pgloader. For posterity, this was my pgloader .load file that ended up working:

LOAD DATABASE
  FROM mysql://hass:password@192.168.50.1:3306/homeassistant
  INTO postgresql://homeassistant:password@192.168.50.1:5432/homeassistant

WITH include drop, create tables, create indexes, reset sequences, workers = 2, concurrency = 1

EXCLUDING TABLE NAMES MATCHING 'statistics', 'statistics_short_term'

CAST type datetime to timestamptz drop default drop not null using zero-dates-to-null 

BEFORE LOAD DO
 $$ drop schema if exists homeassistant cascade; $$, 
 $$ create schema homeassistant; $$;

And a very distant 4th place would be Umami and Gitea tied. Umami because I was unsuccessful (which I guess maybe should make it 5th), and Gitea just because it took forever.

I fully recognize that database migrations aren’t the simplest thing, and the easiest solution would have been to just start fresh, but I really didn’t want to lose any data. Therefore I fought a bit but ultimately seem to have come out (mostly) victorious. The end result is that I only have 2 databases running MariaDB (Booklore and ROMM), and it’s unclear if there will ever be PostgreSQL support for either. But if there is, I will probably migrate.

Also shout out to pgloader as a useful tool for migrating between instances.

I haven’t noticed much performance wise yet (it’s only been a few hours), but I did immediately notice that my database LXC dropped about 5% in RAM usage as a result of the switch. Granted, the LXC only has 3gb of RAM, but still that’s a fairly substantial drop! Will track performance and update this post if I notice anything worth mentioning.

Update: RAM usage seems to now be 50% lower (from ~40% to under 20% overall utilization). And at least HomeAssistant feels snappier in the UI.

How to Enable IPv6 on OPNSense for Matter Support

Sat, 16 Aug 2025 00:00:00 +0000

I’ve wanted to play around with the new hot smarthome standard, known as Matter, for a while. What is Matter? Simply put it’s supposed to be an interoperable standard for all smart home devices. Big companies like Apple, Amazon, and Google are all backing the standard. The hope is to avoid the classic xkcd comic called “Standards”

Personally I’m not fully bought into the Matter standard as I think it has a long way to go and it needs more features/support more device types, though to the standard’s credit they’re continuing to move things along. I’ll put it this way: I’m not going to purposefully no buy a device because it supports Matter, but I’m also not going to go out of my way to buy a Matter-compliant device either. Nor am I going to go out and replace all my devices just to switch to Matter.

I did finally get some Govee LED lights (the Permanent Outdoor LED 2s) and spent a good day and half hanging them up. They support Matter so I figured I would take some time and configure Matter. However it proved to be a bit of an adventure to get it all working because I had to enable IPv6, which is something I had not messed around with before. Nor could I find any good guides for enabling it. Hence this post sharing my experience.

IPv4 v. IPv6

A long, detailed explanation of IPv6 and it’s importance is beyond the scope of this post. In short, we’ve exhausted existing pools of IPv4 addresses (192.168.1.1 is an IPv4 address, for example). IPv4 just uses a set of numbers and therefore only has 4,294,967,296 unique addresses. Sounds like a lot right? But remember with the explosion of cell phones and other smart home devices (as well as the world becoming more connected) there aren’t nearly enough addresses. Hence the development of IPv6, which has 340,282,366,920,938,463,463,374,607,431,768,211,456 unique addresses. That…should be enough for a while.

Matter relies on IPv6 for communication, which is frustrating because I have enough IPv4 addresses internally. But….standards I guess? Anyway, on to how I configured IPv6 on OPNsense.

OPNsense Configuration

The first step is to log into your OPNSense box and go to Interfaces->Settings and make sure the “Allow IPv6” is clicked/toggled on.

Click Save. Not the bit at the bottom that the change will only take affect upon reboot of the machine or reconfiguring each interface. I didn’t need to reboot as we’re going to reconfigure the interfaces.

The next step is to configure/enable DHCP for IPv6 on each VLAN/interface. Personally I have multiple VLANs, but I really only need IPv6 on my IOT VLAN. I also enabled IPv6 on my management VLAN so my network switches and APs would have an IPv6 address. To set up the DHCP, go to the specific interface you want to add IPv6 to. Set IPv6 Configuration Type to Static IPv6. That will add a new dropdown box for configuring DHCP for IPv6. I used the following for my address fd00:0:0:30::1. My IOT network uses VLAN 30, so you can adjust the 30 to be whatever you want. You also need to adjust the 128 to 64 that is in the dropdown next to the address box. Again click Save.

I’m not sure if this next step is necessary because I have VLANs or if it’s optional but I found it mentioned in a few articles/Reddit posts so I figured I would enable it. You’ll need to use “Router Advertisements” under Services and configure it like this:

Essentially I set the “Router Advertisement” setting to “Assisted” and copied the IPv6 prefix from the earlier settings (in my case fd00:0:0:30::1) and clicked Save. You should be able to leave all the other settings the same.

Omada Configuration

I run TP-Link Omada switches/Access Points for my network, and they required a little extra configuration to work as well. Basically I had to go into each network switch, go to “Config”, and scroll down a bit till I saw DHCP options. I needed to enable IPv6 and then I made sure “Dynamic IP (SLAAC/DHCPv6)” was selected. I did similar for each Access Point.

Configuring Proxmox

I had to make a minimal change to my Proxmox network configuration in /etc/network/interfaces to enable IPv6. Specifically I needed to add iface vmbr1 inet6 manual. I use Linux Bridges, so my configuration looks like this:

auto bond0  
iface bond0 inet manual  
       bond-slaves enp4s0f0 enp4s0f1 enp4s0f2 enp4s0f3  
       bond-miimon 100  
       bond-mode 802.3ad  
  
auto vmbr0  
iface vmbr0 inet static  
       address 192.168.1.10/24  
       gateway 192.168.1.1  
       bridge-ports eno1  
       bridge-stp off  
       bridge-fd 0  
       bridge-vlan-aware yes  
       bridge-vids 2-4094  
#Onboard NIC (VLAN 1)  
  
auto vmbr1  
iface vmbr1 inet manual  
       bridge-ports bond0  
       bridge-stp off  
       bridge-fd 0  
       bridge-vlan-aware yes  
       bridge-vids 2-4094  
#LAGG Connection (All VLANs)  
  
iface vmbr1 inet6 manual #THIS IS NEW FOR IPv6!

You’ll need to reboot the server or run sudo ifreload -a to reload the interfaces for Proxmox.

To enable IPv6 for the LXC where I run HomeAssistant, I picked SLAAC.

Now when I go into the LXC (probably need to reboot the LXC), when I run the ip command I see an IPv6 address!

$ ip -6 a show eth0  
2: eth0@if21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link-netnsid 0  
   inet6 fd00::30:893:69ff:fe34:159f/64 scope global dynamic mngtmpaddr noprefixroute    
      valid_lft 86226sec preferred_lft 14226sec  
   inet6 fe80::893:69ff:fe34:159f/64 scope link    
      valid_lft forever preferred_lft forever

In this case, its fd00::30:893:69ff:fe34:159f (notice the 30 from the early step in OPNsense?).

Matter Docker Container

Thankfully this part is pretty easy. The OpenHome Foundation (folks behind HomeAssistant) have made an Docker container to run the server. I suggest running it on the same VLAN as your HomeAssistant instance as I imagine crossing VLANs will be painful. Here’s my Docker-Compose configuration:

 matter-server:  
   image: ghcr.io/home-assistant-libs/python-matter-server:stable  
   container_name: matter-server  
   restart: unless-stopped  
   network_mode: host  
   volumes:  
     - /home/fuzzy/docker/matter-server/data:/data/

Just run docker compose up -d matter-server and it should spin up and hopefully Just Work provided IPv6 is working. You can then add Matter in HomeAssistant (mine popped up automatically when I went to the Devices menu). You’ll have to pair the actual Matter device from your phone as for now Matter also relies on Bluetooth for now (hopefully that will be changing as Matter 1.4.2 was just released that permits WiFi only commissioning).

Conclusion

I may be forgetting a step, but that should get you most of the way there. Not really intuitive at all, but at least now I can connect to Matter devices. But it’s still VERY finicky to get paired into HomeAssistant. Have to disable any VPN, need to be on the same WiFi network, and you sometimes need to factory reset the device/reboot it in order to get it to successfully pair.

The functionality in Matter for the Govee lights is pretty limited, but I’m hoping that improves with time. Still not completely sold on Matter as it’s not very consumer friendly to set up, but again…time.

Setting up Pocket-ID with Caddy-Security

Sun, 13 Jul 2025 00:00:00 +0000

I’ve long been interested in setting up an easy authentication system for logging into (and in some case protecting) services I self-host. I looked into options like Authentik, Authelia, or Keycloak but they seemed overly complicated to me/difficult to set up.

I’ve also wanted to check out passkeys, which are a relatively new form of authentication to replace passwords. They have a lot of potential and I needed something to motivate me to start the transition.

I’ve seen a lot of mentions of Pocket-ID before, which is a lightweight OIDC (OpenID Connect) services that uses passkeys for authentication instead of dealing with passwords. I figured I’d spend an evening spinning it up and seeing if it was as easy as people said it was. Ultimately in some ways it was very easy, but in others I wanted to pull hair out. I decided I should document how I got things working in case it was helpful to others. I’m by no means an expert but I’m happy to help if you run into issues.

Setting up Pocket-ID

Setting up Pocket-ID is really simple. The documentation is actually quite good, though I personally don’t like the .env file approach given how I have my infra set up, so here’s a docker-compose file with the key bits you need:

pocket-id:
  image: ghcr.io/pocket-id/pocket-id:v1.6.2
  container_name: pocket-id
  restart: unless-stopped
  environment:
    - APP_URL=https://pocket-id.example.com
    - TRUST_PROXY=true
    - MAXMIND_LICENSE_KEY=YOUR_API_KEY
    - ENCRYPTION_KEY=YOUR_ENCRYPTION_KEY
    - PUID=1000
    - PGID=1000
  ports:
    - 1411:1411
  volumes:
    - /home/fuzzy/docker/pocketid/data:/app/data

You’ll need a few things to set this up:

Maxmind License Key. You sign up for a key here. I’ve also started using this in conjunction with geoblocking using this Caddy plugin, so it’s worth signing up (and it’s free!).
As explained in the docs, you can set an encryption key. It’s technically optional as you can run Pocket-ID without it, but I’d recommend this step. Security is not a bad idea given that this is a gateway into some of your apps/services.

Also take a look through the various environment variables you can set. Most can also be set from the UI, but I find from a documentation perspective it’s easier to set the variables and then commit it to GitHub so I can easily recreate things if needed.

That’s it! You’ll also need to put it behind a reverse proxy with HTTPS for Pocket-ID/passkeys to work, but that’s beyond the scope of this post. Also consider looking at the hardening guide here.

Configuring Pocket-ID

Apps that Support OIDC

Many apps support OIDC which is an authentication protocol based on the OAuth 2.0 standard. Pocket-ID’s documentation has a number of examples of how to configure apps to work with OIDC, including Proxmox, Nextcloud, Audiobookshelf and others. You can also search for “OIDC NAME-OF-APP” and probably find information on how to set up a few more. In the end a good chunk of the apps I run support OIDC natively, which is great because there’s no additional steps to get things to work. The more complicated part (and what I struggled with) was configuring apps that don’t support OIDC natively.

Apps that Don’t Support OIDC

In the case of apps that don’t support OIDC natively, there are a few options. Again, the Pocket-ID docs on the topic are a great starting point and certainly pointed me in the right direction. Going through the list, I don’t run Traefik (and didn’t want to switch) so that wasn’t an option. I tried both OAuth2-Proxy and TinyAuth. I couldn’t get Oauth2-Proxy to work, probably due to my somewhat complicated setup where I have multiple domains and I use a VPS to tunnel my traffic to home. I liked TinyAuth a lot, but ultimately discovered it wouldn’t work in my setup because I have multiple domains and didn’t want to have to work with multiple instances.

I was about to give up but then I decided to try Caddy-Security. Full disclosure, there were some vulnerabilities disclosed in 2023. The developer of Caddy-Security has responded in this issue on GitHub. You make up your own mind as to whether to rely on Caddy-Security. Personally, I think the security issues are unlikely to affect me in my home-lab, but if this was a corporate thing then it maybe worth closer investigation.

You’ll need to use xCaddy to build your Caddy binary to include github.com/greenpau/caddy-security. That’s beyond the scope of this post as it depends on how you handle your Caddy binary. I use Ansible so it was as simple as adding the repo to the caddy_packages variable. Now comes the tricky part.

The Pocket-ID documentation is…fine but I think left a few things out that took me a while to figure out. The first thing is step 1, creating the OIDC client in Pocket-ID. You don’t need to create multiple clients (unless you really are paranoid/want to). I wanted something simple, so I have just one OIDC client for all of Caddy. To handle that, for your callback URL, you will want to use a wildcard. So instead of setting https://nextcloud.example.com/caddy-security/oauth2/pocket-id/authorization-code-callback and https://homeassistant.example.com/caddy-security/oauth2/pocket-id/authorization-code-callback, you can just do https://*.example.com/caddy-security/oauth2/pocket-id/authorization-code-callback where the * is the wildcard to handle all your subdomains. Make sure you copy/save the client_Id and client_secret, you’ll need those in a minute.

Also, keep in mind pocket-id in the URL is going to be directly tied into the next step when configuring Caddy-Security. So if you change the provider below (I’ll highlight what I mean) you’ll need to adjust your callback URL.

Caddyfile

I’m going to provide the relevant Caddyfile bits now, but scroll down if you want to understand how you can tweak this/things I discovered.

Here’s the top part of my Caddyfile (in the global options block):

Note: Caddy-Security introduced a breaking change in v1.136, released March 1, 2026. The fix is included below, but in short you need to add trust login redirect uri domain suffix example.com path prefix / to the authentication portal.

{
    order authenticate before respond
    order authorize before basicauth

	# caddy-security
    security {
		oauth identity provider pocket-id {
			realm pocket-id
			driver generic
			client_id {{ secret_caddy_pocket_id_user }}
			client_secret {{ secret_caddy_pocket_id_key }}
			scopes openid email profile groups
			base_auth_url https://pocket-id.example.com
			metadata_url https://pocket-id.example.com/.well-known/openid-configuration
			delay_start 3
		}

		authentication portal pocket-id {
			crypto default token lifetime 86400
			enable identity provider pocket-id
			trust login redirect uri domain suffix example.com path prefix /
			transform user {
				match realm pocket-id
				action add role user
			}
		}

		authorization policy pocket-id {
			set auth url /caddy-security/oauth2/pocket-id
			allow roles user
			inject headers with claims
		}
	}
}

Then, to make it easy to import into your existing config, I have the following snippet:

(pocket-id) {
	@auth {
		path /caddy-security/*
	}
	route @auth {
		authenticate with pocket-id
	}
	route /* {
		authorize with pocket-id
	}
}

You can then import this snippet with just import pocket-id like this:

drop.fuzzymistborn.com {
	import pocket-id
	reverse_proxy localhost:3200
}

Explanation/Things I Learned

So what’s different between my Caddyfile and the Pocket-ID documentation? A few things.

I changed the identity provider from generic to pocket-id. Made more sense to me. If you also want to change this value, this is what you need to change in your callback URL above for Pocket-ID. IE https://*.example.com/caddy-security/oauth2/generic/authorization-code-callback versus https://*.example.com/caddy-security/oauth2/pocket-id/authorization-code-callback. You’ll also need to change the realm to match.
In the scopes i added groups which I’ll talk about below.
I changed the authentication portal to pocket-id, again because I think that makes more sense than myportal. I also changed the identity provider to pocket-id instead of generic.
I kept the transform user bit, but there’s another example (in line with #2 above) where you can limit which Pocket-ID groups can log in. I didn’t go this route, but I’ll document a bit how you’d do it if you want.
Again, in the authorization policy I changed things to pocket-id instead of mypolicy and generic.
Finally I used a snippetto make repeating code easier. Repeating myself again, but if you changed anything in the global options block, you’ll need to change the reference in the route @auth and route blocks.

My approach will allow all users in Pocket-ID to be able to authenticate and log into any app/website protected by Pocket-ID and Caddy-Security. For my use case, that’s completely fine as it’s just family using it. But if you wanted to restrict certain pages to certain groups, here’s how you’d go about it.

Restricting to Groups

This post was key to figuring this one out. But essentially the difference between this and the above is that we can create a group in Pocket-ID (and you’ll want to limit the OIDC client as well). In this case we’ll create one called private. Add the necessary users, then create the new authorization policy below. Also, note that the authentication portal is different. We removed this bit:

			transform user {
				match realm pocket-id
				action add role user
			}

That bit is what basically turned all users authenticating via Caddy-Security into a generic user (who could in turn authenticate with Pocket-ID).

Here is what your Caddyfile would look like if you wanted to limit a login to certain groups.

{
    order authenticate before respond
    order authorize before basicauth

	# caddy-security
    security {
		oauth identity provider pocket-id {
			realm pocket-id
			driver generic
			client_id {{ secret_caddy_pocket_id_user }}
			client_secret {{ secret_caddy_pocket_id_key }}
			scopes openid email profile groups
			base_auth_url https://pocket-id.example.com
			metadata_url https://pocket-id.example.com/.well-known/openid-configuration
			delay_start 3
		}

		authentication portal pocket-id {
			crypto default token lifetime 86400
			enable identity provider pocket-id
		}

		authorization policy limited-group {
			set auth url /caddy-security/oauth2/pocket-id
			allow roles private
			inject headers with claims
		}
	}
}

Snippet would look something like this:

(limited-group) {
	@auth {
		path /caddy-security/*
	}
	route @auth {
		authenticate with pocket-id
	}
	route /* {
		authorize with limited-group
	}
}

Bypassing Auth

For some apps/services that have, for example, an API, you may not want to protect that aspect. A great example is HealthChecks. Protecting the /ping endpoint would mean clients wouldn’t be able to just curl the URL, which would basically break the entire point of the app. So I added a custom authorization policy for Healthchecks like this:

authorization policy hc-pocket-id {
	set auth url /caddy-security/oauth2/pocket-id
	allow roles user
	inject headers with claims
	bypass uri prefix /ping/
	bypass uri prefix /api/
	bypass uri prefix /badge/
}

and then a custom snippet (or just included in the actual URL I’m setting up) like this:

(hc-auth) {
	@auth {
		path /caddy-security/*
	}
	route @auth {
		authenticate with pocket-id
	}
	route /* {
		authorize with hc-pocket-id
	}
}

and

healthckecks.example.com {
	import hc-auth
	reverse_proxy localhost:8000
}

healthckecks.example.com {
	@auth {
		path /caddy-security/*
	}
	route @auth {
		authenticate with pocket-id
	}
	route /* {
		authorize with hc-pocket-id
	}
	reverse_proxy localhost:8000
}

The Amazing Arc Pro A40 and Problems with Coral TPU Passthrough

Sun, 13 Jul 2025 00:00:00 +0000

Thanks to a fun garage sale by a certain IronicBadger, I picked up an Intel Arc Pro A40 graphics card for a very reasonable price and decided to put it to use for transcoding. This post is going to be documenting some of my failures/issues in the dream of a single VM to house all my transcoding apps (Jellyfin, Handbrake, and Frigate).

I wrote this post for 2 reasons. First, I wanted to document my failures for the USB Coral TPU passthrough in case someone searching for answers comes across my efforts and learns it’s not going to happen. And second, my experiences so far with the Arc Pro A40 as I’ve been moving my infrastructure around.

TLDR: you can’t pass through a Coral TPU USB device to VM for Frigate, but it was kinda fun trying.

Second TLDR: the Arc Pro is a flipping BEAST for transcoding.

Transcoding Setup

We’ll start with my first attempt to house everything in an LXC. I know there are security concerns/issues with running Docker containers inside an LXC and the Proxmox devs do not support/condone it. But…it works for me and it’s incredibly lightweight so…I do it anyway. However, I ran into an issue that I couldn’t get around. Namely that was passing through the correct graphics card to the LXC as I have both an iGPU from my i5-10400 and the Arc Pro. Annoyingly in my past experience the two would flip back and forth between /dev/dri/renderD128 and /dev/dri/renderD129. Therefore I couldn’t reliably ensure that the Arc Pro was the card available in the LXC, which kind of defeated the entire purpose of the setup. I only wanted to pass the one through so that way when I configured the services I knew I had the Arc Pro available and not the iGPU.

So after a few days spent toying with udev rules, I abandoned the LXC approach and went for a full VM. Much simpler to handle passing only the Arc Pro through and the trade off of the higher resource requirements was worth it for a guarantee of always having the Arc Pro available. All my issues solved right? Wrong.

Coral TPU over USB

As I mentioned above, I wanted to also run Frigate in the VM and use the Arc Pro to handle all the transcoding of my camera streams. I use a Coral TPU over USB, which has worked extremely well as an efficient solution for person/car detection in Frigate. I figured it wouldn’t be hard to pass through the USB device to the VM. How wrong I was.

I quickly learned how much of a special snowflake the Coral TPU USB is. When you first plug it in, it shows up as

Bus 002 Device 002: ID 1a6e:089a Global Unichip Corp.

However, after it’s used/initialized, it changes it’s device ID (!!!!!!) to:

Bus 002 Device 003: ID 18d1:9302 Google Inc.

What. The. Hell.

You can read more about it in this long GitHub issue.

Because it changed IDs (and I think I tried passing both through to the VM and it didn’t work), I needed a way initialize the Coral on the host then pass through the “right” device ID to the VM. So along with my AI pal Claude, we came up with this solution:

Step 1: Created a Docker container that contains the necessary bits to intialize the device: https://github.com/FuzzyMistborn/docker-coral-usb/?tab=readme-ov-file

Step 2: Made a bash script that would run the Docker container:

#!/bin/bash 
docker run --rm --device /dev/bus/usb ghcr.io/fuzzymistborn/docker-coral-usb

Step 3: Create a udev rule to run the bash script when the Coral is plugged in:

ACTION=="add", SUBSYSTEM=="usb", ATTRS{idVendor}=="1a6e", ATTRS{idProduct}=="089a", RUN+="/home/fuzzy/usb-docker-trigger.sh"

Step 4: Create a systemd service to also call the bash script on boot:

[Unit]
Description=USB TPU Initialize Docker on Boot
After=docker.service
Wants=docker.service

[Service]
Type=oneshot
ExecStart=/home/{{main_username}}/usb-docker.sh
User={{main_username}}
RemainAfterExit=yes
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Step 5: Profit?

That all worked. I could see the Coral TPU in the VM as the Google Inc. whenever I would reboot the server or plug/unplug. Problems solved right?

Wrong.

I noticed very quickly that my Frigate logs were filled with this:

frigate.watchdog               INFO    : Detection appears to be stuck. Restarting detection process...
root                           INFO    : Waiting for detection process to exit gracefully...
root                           INFO    : Detection process didn't exit. Force killing...
root                           INFO    : Detection process has exited...
detector.coral                 INFO    : Starting detection process: 34258
frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

I found a number of threads like this one or this oneof people with the issue. I tried all the solutions: a powered USB hub, a shorter cable, etc. None of that worked. I thought about getting the Dual PCIe TPU instead, but that was quickly going to add up in costs, partly thanks to tariffs and also partly just because they’re hard to find as I believe Google has stopped production.

For now, I’ve reverted to having Frigate run on the host in Docker with the TPU USB passed through, and that works fine. I’ve been playing with OpenVINO on the Arc Pro in the VM and it consumes a lot more power in comparison to the USB Coral. So I think until the USB Coral dies, I’m going to stick with my current setup as the power draw is well worth it. Plus I have the detection settings dialed in nicely for the TPU and I would have to play a bit more with OpenVino to get it “right.”

Arc Pro Transcoding Performance

It’s. A. Monster.

As mentioned above, I have an i5-10400 in my server, which has a UHD Graphics 630 iGPU. Per the very handy Wikipedia page on Intel Quick Sync, the 630 can handle HEVC and even HEVC-10bit. It cannot handle AV1, which the Arc Pro can.

I haven’t really pushed the Arc Pro long term yet, but with some quick performance tests in Jellyfin, I found the Arc Pro was about twice as fast transcoding 4k files than the iGPU. Granted, we’re talking ~60 FPS on the iGPU so it wasn’t BAD, but I’m never going to have transcoding issues/stutters with the Arc Pro.

The addition of AV1 also is potentially interesting, though I noticed the speed is more like 25% faster on the Arc Pro than the iGPU (50fps on the iGPU, 75fps on the Arc Pro). What’s more interesting is that when I used Handbrake to reencode some files, the resulting files were noticeable smaller from the Arc Pro. For example:

Device	Starting Size	Ending Size
iGPU	8.7 GB	3.9 GB
Arc Pro	8.7 GB	2.7 GB

In terms of power draw, I’m relatively pleased. My first experience with an Arc card, the A380, the power draw was terrible. I think there have been some improvements to the Linux kernel and also more time to figure out that you need to enable ASPM in the BIOS. Thankfully my motherboard supported ASPM so I was able to get the draw down to around 10watts idle.

I think long term the Arc Pro is going to have a happy life in my home lab. One day it’ll probably take over image processing for the Coral TPU while also handling all my transcoding needs. And who knows what else some clever developer will come up with that can take advantage of the Arc card.

How to Send an Snapshot of a Traffic Map to Your Phone

Sat, 25 Jan 2025 00:00:00 +0000

One of my absolute favorite automations I have set up is the one that sends me a notification on workdays with an image displaying a Google Map with the traffic layer turned on. Thankfully I have several ways I can get from home to the office, but the main way along the interstate backs up very quickly if there’s an accident or other issue.

I’ve been using Google Maps for a while, but this solution should work for any traffic map you can find that allows you to set the lat/long/zoom by URL. I also just started playing around with using the Waze map to send myself a screen shot of the possible routes home. And the other benefit is I can use CATT in the morning to cast the map to the Google Home Hub that sits in my kitchen so I can see my commute as I’m eating breakfast and making my lunch.

Under the hood this relies on Selenium and Splinter, which is a Python app that has a relatively straightforward API to interact with Selenium.

Install Splinter

You’ll want to install Splinter in a Python virtual environment, particularly if you’re running recent versions of Ubuntu or Debian. pipx won’t work. You can read more about virtual environments here, but essentially it’s like a mini virtual machine but just for python, allowing it to be isolated from the system’s python packages.

I set my venv up in .venv, so the command to create it for me was:

python3 -m venv .venv

You can use whatever directory/path you want though. Once that’s done, you need to activate the virtual environment with this (adjust the .venv part at the beginning depending on the path you selected above):

source .venv/bin/activate

Once the virtual environment is activated, run this command to install Splinter:

pip3 install splinter[selenium]

This also will install the needed driver/bindings for Selenium.

Install Selenium

For running Selenium, I found the available docker container to be easiest. I’m a fan of Firefox, so I chose the Firefox version but you could probably do this with Edge or Chrome as well. Here’s my docker-compose snippet:

services:
  selenium:
    image: selenium/standalone-firefox:133.0
    container_name: selenium
    ports:
      - 4444:4444
    environment:
      - SE_NODE_MAX_SESSIONS=2
      - TZ=SET_YOUR_TIMEZONE
    shm_size: 2g
    restart: unless-stopped

You’ll want to update the tag to be whatever version you want. I’d recommend running the latest version (you can read more how I handle updates to my infrastructure here), but not the latest tag. Also set the timezone as that will be helpful if you also want the Waze directions option discussed below.

I did notice a bit of a memory leak with Selenium, so I scheduled a daily reboot at 4:30am via cron with the following:

30 4 * * * docker container restart selenium > /dev/null 2>&1

Google Maps Setup

You can skip this part if you don’t want to use Google Maps. I understand the want/desire to deGoogle, but I still find Google Maps to be unparalleled, particularly for traffic data. Since the only thing Google is getting is the location of the map I want to generate, I think the trade off is worth it, but again, entirely up to you.

The first thing you’ll need is a Google Maps API key. You can follow the instructions here. We’ll be using the Maps JavaScript API. You do need to provide credit card information to access it, but there is a free tier that you should be able to stay well under. Google is making changes to their API pricing, but from what I can gather, I don’t think the changes impact the Maps JavaScript API. You only get charged an API hit when you load the map, so based on the cronjob I have detailed later, I only use ~1,000 API requests a month. I’m sure I could streamline that down further if needed. If you’re worried about being charged, you can set quota limits.

Now that you have the API key, you’ll need to have a way to display the map. Below is some HTML that will render the map in a browser.

<!DOCTYPE html>  
<html>  
 <head>  
   <meta name="viewport" content="initial-scale=1.0, user-scalable=no">  
   <meta charset="utf-8">  
   <title>Traffic Layer</title>  
   <style>  
     /* Always set the map height explicitly to define the size of the div  
      * element that contains the map. */  
     #map {  
       height: 100%;  
     }  
     /* Optional: Makes the sample page fill the window. */  
     html, body {  
       height: 100%;  
       margin: 0;  
       padding: 0;  
     }  
   </style>  
 </head>  
 <body>  
   <div id="map"></div>  
   <script>  
     function initMap() {  
       const map = new google.maps.Map(document.getElementById("map"), {  
         zoom: YOUR_VALUE,
         center: {lat: YOUR_VALUE, lng: YOUR_VALUE},  
         disableDefaultUI: true  
       });  
       const trafficLayer = new google.maps.TrafficLayer();  
       trafficLayer.setMap(map);  
     }  
  
     window.initMap = initMap;  
   </script>  
   <script async defer  
   src="https://maps.googleapis.com/maps/api/js?key=INSERT_YOUR_API_KEY_HERE&callback=initMap">  
   </script>  
 </body>  
</html>

You will need to insert your API key towards the bottom (replace INSERT_YOUR_API_KEY_HERE). You’ll also need to set the desired latitude and longitude, as well as the zoom level (replace YOUR_VALUE, from my experience something like 10-13 gets you into the ballpark you probably want). Save it as an html file, like traffic.html.

To view the page there are a few options. The option I used for a long time was to copy the HTML file into the www directory in HomeAssistant and accessed it via http://YOUR_HA_INSTANCE_IP:PORT/local/traffic.html. However, if your HA instance is publicly accessible, so is your map. Which isn’t ideal, because 1) bots could abuse it and rack up a bill if you didn’t set a limit and 2) well…it’s public so anyone could see roughly where you live if you set the view around your house. The other option is to use a reverse proxy, like Caddy, nginx, or Apache (I think Traefik can handle static files but I’m not 100% sure). Here is an example of my Caddy file:

traffic.example.com {  
       file_server  
       root * /home/fuzzy/web  
       tls /home/fuzzy/lego/certificates/_.example.com.crt /home/fuzzy/lego/certificates/_.example.com.key  
}

Note that the TLS/HTTPS part is optional. I’ve used AdGuardHome to rewrite URL queries to a custom domain pointed at my Caddy instance. So HomeAssistant and other things on my network can access it, but it’s inaccessible outside my LAN.

You’ll end up with a screen shot that looks like this:

Waze Setup

If all the above seems way too complicated to you (API keys, html pages, reverse proxies oh my!), there’s another simpler option. Waze offers a live map that shows real time traffic info (yes I know Google owns Waze, but I haven’t found another traffic map with programmable URLs, let me know in the comments if you know one). The URL is this:

https://embed.waze.com/iframe?zoom=YOUR_VALUE&lat=YOUR_VALUE&lon=YOUR_VALUE&ct=livemap

Just like with the Google map, play around with the zoom level (from my experience something like 10-13 gets you into the ballpark you probably want) and location latitude/longitude. This is the result:

Alternatively, you can use the Driving Directions map to generate a map. The URL looks something like this:

https://www.waze.com/live-map/directions/the-metropolitan-museum-of-art-5th-ave-1000-manhattan?utm_campaign=iframe_search&utm_medium=fullscreen_map&utm_source=direct_link&to=place.w.187433368.1874595822.373029&from=place.w.187433367.1874399211.868693

And the result is this:

Putting it All Together

Now that we have all the pieces, time to actually get it to work in HomeAssistant! The first thing you’ll need is a Python script to take the snapshot. Here’s an example for the Google Map option:

from splinter import Browser  
  
browser = Browser(driver_name='remote',  
   command_executor='http://SELENIUM_IP_ADDRESS:4444/wd/hub',  
   browser='firefox',  
   wait_time=20  
)  
browser.visit("https://traffic.example.com")  
  
# Solution to fix "wait" which is broken  
browser.find_by_css('Test')  
  
browser.screenshot("/home/fuzzy/hass/tmp/commute", full=True, unique_file=False) 
  
browser.quit()

And the Waze LiveMap option:

from splinter import Browser  
import time  
  
browser = Browser(driver_name='remote',  
   command_executor='http://SELENIUM_IP_ADDRESS:4444/wd/hub',  
   browser='firefox',  
   wait_time=20  
)  
browser.visit("YOUR_WAZE_URL")  

# Hack to wait for JavaScript to fully load
time.sleep(10)  
  
browser.screenshot("/home/fuzzy/hass/tmp/waze", full=True, unique_file=False)  
  
browser.quit()

And finally, the Waze Directions option:

from splinter import Browser  
  
browser = Browser(driver_name='remote',  
   command_executor='http://SELENIUM_IP_ADDRESS:4444/wd/hub',  
   browser='firefox',  
   wait_time=20  
)  
browser.visit("YOUR_WAZE_URL")  
  
# Solution to fix "wait" which is broken  
browser.is_text_present("Arrive")  
browser.find_by_xpath("/html/body/div[1]/div[2]/button").click()  

browser.screenshot("/home/fuzzy/hass/tmp/waze", full=True, unique_file=False)  
  
browser.quit()

FYI the xpath bit clears out the Waze banner at the bottom about installing the Android/iOS app.

You’ll want to adjust the visit URL as well as the Selenium IP address and screenshot save path. Note you do not need to put a file extension for the screenshot path. In my example, it will save the screenshot as /home/fuzzy/hass/tmp/commute.png or waze.png. I’d suggest putting it in a subdirectory in your HomeAssistant folder to make the next part easier. Make the .py file executable: chmod +x YOURFILE.py.

To take the screenshot, I use two cronjobs. The first handles snapshots during commute times, the second during times surrounding commute time

*/10 7-8,16-18 * * 1-5 .venv/bin/python3 /home/fuzzy/scripts/traffic.py
*/30 9-15 * * 1-5 .venv/bin/python3 /home/fuzzy/scripts/traffic.py

You’ll need to update .venv/bin/python3 to be whatever path you set up your venv in above. Basically the first cronjob updates every 10 minutes between 7 and 8am and 4 and 6pm Monday through Friday, and the second runs every 30 minutes between 9am and 3pm Monday through Friday. IE it updates more frequently during the times I’m most likely to commute, and less often during the workday in case I leave a bit later or earlier. And then it doesn’t run any other time. This is how I greatly limit the number of API calls. You can use https://crontab.guru to come up with your own setup.

HomeAssistant

After all that work, you finally should have a snapshot of whatever map you chose saved in a directory HomeAssistant can access. You can then use this to send a notification to your phone as desired. I also add a little spice by adding the following text beneath the image:

Here is your current commute. The fastest route takes approximately XY minutes via I-95.

You can get the “via” part if you use the Waze Travel Time integration though I’ve found it to be slow to update. Here’s an example of how it looks in Signal:

Again, over the years I’ve found this to be invaluable for knowing both how long my commute is going to take and if there’s going to be traffic and I need to find a different way home. I hope you find it useful as well!

2024 Homelab Remodel

Tue, 11 Jun 2024 23:00:00 +0000

I recently finished some homelab renovations, both in hardware and in software, that I think are pretty neat. This post will be a quick summary of some of the bigger changes.

No More Drone

Continuing on my love/hate relationship with Drone, I finally hit an overwhelming sense of hate, so that’s where I"m going to start. Part of the reason why it’s been a while between posts (Ok ok, I’ve also just been super busy) is that my automated build setup for deploying this blog to my VPS has been annoyingly broken due to this issue. And over a year later there’s been no resolution (and no other commits to the repo/image). Now granted, as I’ve now discovered, I could use just any Docker image with Hugo I wanted to generate the blog, but I think this issue underscores an issue I’ve had with Drone, namely that there hasn’t been anything new or added in the almost 3 years I’ve run it.

The positive way to look at that is that there hasn’t been anything that’s particularly broken about Drone in that time. But I think it’s representative of a larger issue, namely that they no longer support the open source version. There’s also this:

Whenever I logged into Drone, this banner was stuck on the bottom. No way to hide or remove. And more annoyingly, IT BLOCKS PEOPLE FROM SEEING THINGS IN THE UI! Plus, I’m self-hosting Drone, which should be free (and as best I could tell, there was no limit on what I could do, beyond the banner). So while not itself a dealbreaker, I was reminded every time I logged in that Drone is basically dead for self hosters.

So I decided to take another look at Woodpecker CI. And wow things have improved over the past few years. I was able to move all my repos over to Woodpecker over the course of 2 nights, largely helped by this helpful plugin to convert Drone pipelines to Woodpecker. I was able to take the converted files and with a little trial and error (plus some tweaking because things had broken over the past few years) I was able to fully convert off Drone. Honestly the part I struggled with the most was getting the agent/runner on my local server to connect to the Woodpecker server running on my VPS. But in the end, I’m happy with the change and to be running an actively developed project. So if you’re running Drone still, I’d highly suggest checking out Woodpecker.

Now why not GItea Runner you may be asking? I looked hard at it, but what ultimately held me up were 2 things. First, my VPS lacks a lot of horsepower (on purpose) so I’d prefer the build agents to be on my local server. While I understand I can run the Gitea runner/agent locally I tried it and it didn’t seem to be the easiest thing to run. More importantly, I would have had to move all my development over to Gitea instead of Github. I wasn’t keen on the idea, largely because I view my Gitea instance as a testbench and as a backup/mirror of my work. Not the primary source. So it would have been a fundamental change in my setup that I’m not quite ready to take. Not knocking Gitea at all here, it’s just not what I’m after right now.

Goodbye Synology

In hardware upgrades, I realized that my Synology NAS was…well….old. It’s a DSM 713+, meaning it’s from 2013. The Linux kernel on the thing was 3.10, I’m pretty sure it wasn’t getting updates anymore….it was time. I recently sunset my BlueIris box in favor of going all in on Frigate (which has been amazing and it looks like the next release in 0.14 is going to make some major UI improvements I’m eagerly awaiting), so I realized I had a spare i5-10400 lying around doing nothing. I decided to use this as the basis for a new NAS box. And I decided I wanted to make it rack-mounted. I ultimately landed on this RackChoice 2U server chassis. Overall, I’m going to say I’m impressed with it. The build was a little tight in some spots and cable management was….well…basically non-existent. I largely just tried to make sure the airflow wasn’t too obstructed.

I also had to deal with a dead power supply which was annoying (the BeQuiet in the above picture, swapped it for a Corsair RM750 which works just fine). The only other hardware change I made was I swapped the 4 front mounted fans for some Noctua NF-R8’s. The included fans were probably fine, but I’ve always loved Noctua fans for their performance and silence. And they weren’t too expensive.

I also dealt with some tight clearance in the back of my rack (19" wall mounted). I accidentally squeezed the box’s networking cable (still works just fine but…it’s on the “to replace” list).

Software wise, I went with Proxmox and created a cluster with my other rig, so now I can move LXCs/VMs around as needed for maintenance if needed. Plus, I have a ton more horsepower at my disposal. I’ve got 3 disks in the 2U chassis (with room for 2 more if needed, which I probably won’t), running with MergerFS and Snapraid to create a main volume of about 12TB. Plenty for some on-site backups/redundancies.

Here’s the crazy part though. I swapped the Nvidia Quadro P400 from my main server rig into this new box (while adding a 3060 to the other box for AI/Immich photo processing, more on this another time). All of that resulted in a power load that basically matched what the Synology was using. I’m calling this a big win overall.

LNXLink

The last thing I want to mention is a very handy piece of software if you’re running Linux on your desktop and want to get data into HomeAssistant: LNXLink. If you ever used HASS.Agentor IOTLink on Windows, this is basically the Linux equivalent. All communication is sent over MQTT, which makes it very easy to import into HomeAssistant. Versus before I was using Telegraf to export data to Influx and then had to define all these sensors to get data back into HomeAssistant. Extremely convoluted and not easy to manage.

I’ve cobbled together a somewhat basic install role via Ansible that also allows me to dynamically set the configuration for each computer. And I have it running on several of my LXC’s (as well as the main Proxmox hosts) reporting data back to HomeAssistant that then gets nicely displayed.

On top of my Telegraf-Influx-HomeAssistant setup, I was running an app called Webhook that allowed me to execute scripts/commands on some of my hosts. Things like rebooting the computer, running a bash script to display traffic information for my commute, starting/stopping a VM, etc. LNXLink has a Bash module that lets you define commands that can be executed with a push of a button. Obviously, be careful doing this since the commands can be executed via MQTT. But it removed yet another piece of software for me to keep up/manage. And the icing on the cake is that the developer is super responsive and willing to add new features. Really awesome piece of software.

Conclusion

I’ve made a few other changes in my digital life over the past few weeks (moved off Arch to Fedora, got a new Framework laptop, started messing with AI/Immich) which maybe I’ll cover in more detail later. But for now those are the big things that have sucked up my time lately in my #HomeLab.

Bye Bye Minio, Hello Restic Rest Server!

Thu, 06 Jul 2023 11:01:00 +0000

Previously, I wrote about Minio, an S3 compatible object storage solution, as the backend for backing up my data locally using Restic. And largely it worked great. I never had an issue where data wouldn’t be sent or anything along those lines (which ultimately is what matters here). But I had many, many issues when it came time to update.

Minio Issues

Admittedly, I was very infrequent with my updates (anywhere from 6 months to a year). Largely that was because I’d forget I was running Minio as it 1) just kept working and 2) since I rarely log into my Synology box I’d forget about updates. Plus, since I don’t keep my Synology docker-compose file in my infra repo, I don’t get automatic update notifications like I do with Rennovatebot. This definitely is largely a “me” problem, but let’s talk about the updates themselves.

Whenever I would update, I’d usually run into a massive breaking change that would take hours (sometimes over multiple days) to resolve. And this last time the breaking change was a doozey. As best I understand it, they deprecated the underlying filesystem used by default with the Docker container sometime in 2020. And as of a release in October 2022, the code was removed. That’s fine, I usually don’t have an issue with deprecations. However, what was absolutely maddening about this change is that there was no automated migration for users, and in fact, the ONLY way to update is to spin up a new instance of Minio and COPY ALL THE DATA over with various commands (and basically going over HTTP). You can’t just spin up a new instance and point it at the older folder structure/data.

I also ran into a number of other issues trying to follow the documentation, namely the mc binary isn’t included in the Minio server docker container, so I had to find a way to install the binary. Then there were some version incompatibilities I had to deal with. And then the final nail in the coffin was when I ran the mc mirror command. A number of errors popped up that gave me serious concerns as to whether the data was being transferred over. That left me with 2 options: 1) spin up a new Minio instance and forget all the data, and maybe find myself in another upgrade hell situation in another year or 2) still forget about the data but find a new solution. I chose #2 because fool me once, shame on me. Fool me twice, I ain’t gonna let it get to thrice.

Minio Alternative - Restic Rest Server

The solution I came across thanks to user @TheDragon on the Self-Hosted podcast Discord server is a rest server created by the restic devs. It is a very lightweight server that basically provides a way for data to be transferred over HTTP. And while Minio wasn’t resource heavy, I didn’t really need S3-compatible object storage for just backing up my data. I just wanted an easy way to send my restic backups to a different host without having to use SSH/FTP.

It’s really easy to spin up. Here’s my docker-compose:

  restic-server:
    image: restic/rest-server:0.12.0
    container_name: restic-server
    volumes:
      - /YOURMOUNTPOINT:/data
    ports:
      - 8000:8000
    restart: unless-stopped

Yep, that’s it! Once the server is up you create a user with docker exec -it restic_server create_user myuser and pick a password. It was also pretty easy to migrate my Autorestic configs as the Rest server is a supported backend.

I did toy around with the idea of running the rest server on my main proxmox host and have the folder mounted over NFS. However, upon reflection I realized that was probably a poor idea (and in fact when I tried it the logs kept nagging me that it was a bad idea). If there ever was a disconnect (for whatever reason) between the host and my Synology the data wouldn’t get backed up. Or well…it would but on my host which isn’t what I want. So instead I set up the rest server on my Synology. To get update notifications I’m tracking the Github releases page and also left the docker-compose snippet in my Proxmox host’s config (without running it) so that way Rennovatebot will also find the changes.

Final Thoughts

Some other fun things for those who might be interested is that you can run the rest server in “append only” mode which prevents deletion/modification of existing backups. Handy if you want to prevent any kind of ransomware infecting your backups. Also, there’s a way to export Prometheus/Grafana data for you data nerds out there.

And in terms of my old data, I left it on my Synology for now, and if I find myself needing it in the next few months I can spin up Minio for a minute to pull the data. And after a few months, I’ll just delete the data.

So if you’re using MInio just for backups, take a look at the restic rest server. It’s much simpler to spin up, works just as well, and (hopefully) shouldn’t be nearly as much of a PITA to update!

Permanently Mounting Cameras without Damaging House

Tue, 07 Mar 2023 23:01:00 +0000

I still intend to do a write up on my choices for cameras at the new place, but in the meantime I wanted to do a quick bit on how I mounted cameras outside. At my old house, I’d drilled/screwed into the siding/soffit but I don’t like how that leaves holes if/when you move. I was keen to find a solution that was solid without leaving lasting damage. After some searching, I found a few ideas, including siding clips and aluminum strip slotted under the eaves. The clips looked interesting, but I was worried about how they’d hold up in the wind. I was intrigued by the aluminum strip idea and it looked pretty simple. Credit to this video for the idea.

All you need is some aluminum bar, and if you really want it to blend in, some spray paint that matches your siding/soffit color. You’ll also need a saw and a drill, and screws/washers that fit your camera mount. For me that was #8 machine screws.

Figure out where you want to mount the cameras and measure the width of the eave. After you cut down the strip, take the camera mount and trace the mounting holes. Don’t worry about being too close to the edge, the aluminum should hold pretty well. Drill through the aluminum (be patient and take your time here) and check your mount. At this point I took the strip and spray painted it white.

After you’re done, it should look something like this:

I did find that after I had screwed everything through, I needed to cut the screws down to the nut, otherwise they stuck out too far/scratched the soffit. Nothing a hack saw can’t solve, but again take your time.

When it came to mounting below the eaves, it was pretty easy. Just stick one end in and then slide the other end in on an angle. The hard part is getting the ethernet cable through after that. It should be prety easy from there to adjust where the camera is pointing and get the view you want.

And here’s a closer look:

So far the cameras haven’t moved or fallen at all, and it gets windy around here. Haven’t seen if they’ll stay up in a hurricane or anything like that, but 20-30 mph winds are no problem. If you want, you could use silicone or screws but I haven’t found that necessary.

HomeAssistant App and TinyCam - Powerful Combination!

Sun, 19 Feb 2023 23:01:00 +0000

This is going to be a quick one, but I’m unreasonably excited about it. Lately I’ve noticed that image-based notifications haven’t been always going through via Telegram. Lots of errors in the logs about timed out connections, which I chalk up to the fact my internet is over a 5G router. It’s not bad, but it’s not always stable/reliable. I started to consider alternatives like Gotify (which I already use, except not for images) or even something Matrix. But then I decided to take a closer look at an app I already have installed, the HomeAssistant companion app!

The HomeAssistant companion app is way more than just a way to access HomeAssistant. It can provide all kinds of sensor data about your phone back to HASS and also serve as a vehicle for notifications. Not all of which need to be an actual message! You can do all sorts of things, like turn Bluetooth on/off, set do not disturb mode, and even trigger other apps to do things (I believe this part is Android only, I don’t use iOS so apologies to those who do).

When I saw that it was possible to alter the action when you click the notification I was intrigued. Then I saw that you could also call an Android intent to open up another app. And somewhere I saw the idea to open up TinyCam and I had to dig in. I’ve been using TinyCam as a quick and easy way to access my POE cameras as I have a mix of camera brands (post coming soon, promise). I tried the BlueIris app and it’s OK, but the UI isn’t ideal for quickly seeing a camera feed. TinyCam on the other hand literally just shows the camera feeds on the opening page. Doesn’t get much simpler.

I had a “ah-ha” moment realizing I could use the HomeAssistant companion app to send a notification with a snapshot of the camera feed when motion is detected, and then when I click the notification, it takes me directly to that camera’s feed in TinyCam so I can see what’s going on. I struggled for the better part of a day to figure out the right intent (hat tip to the HASS Discord server and @dshokouhi, as well as /u/nVIceman on Reddit) but I got it. So now if there’s motion at my front door, I get a notification with the doorbell image and if I click the link, the doorbell camera feed opens in TinyCam.

service: notify.mobile_app_fuzzy_phone
data:
  message: Motion detected at the Front Door!
  data:
    image: /api/camera_proxy/camera.truthwatcher_front_door
    clickAction: "intent:#Intent;action=android.intent.action.MAIN;component=com.alexvas.dvr.pro/com.alexvas.dvr.activity.LiveViewActivity;S.com.alexvas.dvr.intent.extra.shortcut.NAME=Front Door;end"

That’s it. All you’d need to do is change the last bit (after “NAME=”) to match the name of the camera in TinyCam. If you wanted instead to open the camera directly without having to click a link, you could do something like this:

service: notify.mobile_app_fuzzy_phone
data:
  message: command_activity
  data:    
    intent_action: "android.intent.action.MAIN"
    intent_extras: "com.alexvas.dvr.intent.extra.shortcut.NAME:Front Door"
    intent_package_name: "com.alexvas.dvr.pro"

In closing, I’d encourage everyone to take a close look at the HomeAsssistant companion app docs. It’s really impressive what it can do!

MyQ - Local Garage Door Control

Thu, 16 Feb 2023 11:01:00 +0000

EDIT 11/11/2023: Chamberlain has announced they’ve intentionally broken all third party integrations that don’t pay them, claiming they are preventing “unauthorized access.” As a result, HomeAssistant is going to be removing its MyQ integration in the 2023.12 release. Since this post, I’ve become aware of Ratgdo, which is an ESP based project which directly integrates with the garage opener and in effect emulates being a MyQ/Security 2.0 device. I bought one and have had it installed for about a month, and quite frankly it’s awesome. It allows for a lot more nuanced control (such as opening the garage to a set position, and also the number of times the door has opened/closed). Plus it just looks a lot cleaner than the below solution. At this point I would highly recommend Ratgdo over the below solution, but the below still works if you are interested.

Unfortunately, when we built our new house, the options for garage door opener were: MyQ, MyQ and MyQ (aka there were no options). MyQ is made by Chamberlain (who also makes LiftMaster products), which basically is the garage door opener market. There are a few other brands out there, but mostly all I see these days is Chamberlain. Why don’t I like MyQ? Well, putting aside that it’s a cloud-dependent garage door opener, I’m not a fan of Chamberlain’s business practices. In the past, they’ve tweaked their API with the apparent purpose of preventing third party integrations like HomeAssistant. And also, they’ve charged for integrating MyQ with Alexa and Google Home. Which just feels….dirty.

Personally I would have preferred a “dumb” garage door opener that I could then have smart-ified as I had before with a Zwave-based GoControl/Linear opener. But I didn’t feel like ripping out a perfectly good garage door opener and replacing it to the tune of a few hundred dollars. So alternatives had to be found. Unfortunately, Chamberlain has implemented something they call “Security 2.0.” As I understand it, in “ye olden days,” when you hooked up a button to a garage door and pressed the button, it basically shorted the connection and that caused the door to open or close. Security 2.0 adds “security” by sending a rotating code to the door. I honestly don’t know any security this actually provides, but it does mean that the only devices that can work to open/close a MyQ garage door have to be made by Chamberlains (bet you didn’t see that coming….).

So how to work around it? Well thankfully some clever people figured out that you can solder some wires to a cheap Security 2.0 button and hook it up to a smart relay (and the garage door). Then all you need to do is trigger the relay, which “presses” the button, and then boom, open garage door!

Here’s what you need:

a Security 2.0 controller. You can use a wireless remote if you want, or a button like this for $10-12 on Amazon. Don’t feel like soldering? You can buy one from Garadget for $20ish (plus shipping).
some kind of smart dry contact relay. Personally I picked up a Zooz Zen16 Multirelay but something like a Shelly should work too.
wire (doorbell wire is fine), solder, and a solder iron
Optional (and highly recommended) - some kind of tilt sensor so you can monitor whether the garage door is open or close. I used a Zooz XS Tilt/Shock sensor. Note that Zooz sells this as a kit along with the multirelay here.

I forgot to take a picture of where exactly to solder, but here’s a good thread from the Hubitat forum. As you can see next to the button there are 4 contacts. You need to solder two wires to the board (IIRC you need to pick a side, so if holding the board vertically, either the top or the bottom). Then connect more wire to the screw terminals. Basically think of the button as a pass through. You are going to use your smart relay to trigger the button, which then in turn will trigger the garage door.

Take that to the garage and wire in your smart relay (the wires you soldered onto the button). That will depend on what relay you picked. For me, mine looked like this:

Last step is to wire the button with the wires coming off the “normal” button terminals into the garage door opener. I just stuck mine on the top of the garage door opener as it gave me convenient access to power for the relay and my wire runs didn’t have to be that long.

At this point I’d suggest testing your relay to make sure it all works. To get the garage door into HomeAssistant I used a Template cover, using the tilt sensor for the value_template (and also throwing MyQ in there for the “opening” and “closing” as a “why not”) to track whether the door is open or closed. Then the open/close is just a trigger to the relay. Here’s my template:

- platform: template
  covers:
    garage_door:
      device_class: garage
      friendly_name: "Garage Door"
      unique_id: "garage_door_template"
      value_template: >
        {% if states('cover.garage_door_myq') == "opening" or states('cover.garage_door_myq') == "closing" %}
          {{ states('cover.garage_door_myq') | capitalize }}
        {% elif states('binary_sensor.garage_door_tilt') == 'off' %}
          Closed
        {% elif states('binary_sensor.garage_door_tilt') == 'on' %}
          Open
        {% else %}
          N/A
        {% endif %}
      open_cover:
        - service: switch.turn_on
          target:
            entity_id: switch.garage_door_opener
      close_cover:
        - service: switch.turn_on
          target:
            entity_id: switch.garage_door_opener
      stop_cover:
        - service: switch.turn_on
          target:
            entity_id: switch.garage_door_opener

I did need to do some adjusting to the Zen16 configuration. I set the Switch Type to “Momentary” (not “Garage Door momentary” as that didn’t work for some reason) and set an auto turn off for the relay (I set mine to 1 second). You may need to do something different.

This was actually a pretty easy thing to set up, and it is so nice to have local control of my garage door and not be dependent on MyQ (while I guess still having access to MyQ if I still need it for whatever reason).

New House - New Smart Home Hardware!

Wed, 04 Jan 2023 11:01:00 +0000

I realize in hindsight I didn’t post much in 2022. Well it’s 2023 now and I guess you could say I have a new year’s resolution to post more. But in reality, the reason why I didn’t post much last year was because I was moving. No, it didn’t take me all year to move. But we decided to build a new house after searching a bit and the whole build process took about 9 months start to finish. And while the build was happening, I had zero interest to make major changes to any of my home automation or networking setup. Instead, I focused my time on planning out hardware in the new place as after about 5 years of a smart house I’d learned a few things. I’m not at all close to done with getting everything set up automation wise, but I am 95% done with the installation. I thought now would be a good time to start documenting the new gear. This will be in two parts. Part 1 will focus on the smart home gear and part 2 will focus on my networking setup.

If you want to look at what I had in my old house, you can see the post here

Here’s an image of all the stuff I’ve deployed (may be a few things missing, and things I’ve reused aren’t pictured)

Thermostat

In the old house, I ran an Ecobee 3 for the entire time I lived there and I was pretty happy with it. While it’s cloud connected (and a few years ago Ecobee had more than it’s fair share of server outages), lately it’s been very reliable. That being said, I did strongly consider going Zwave for the thermostat, specifically the Honeywell T6 Pro. It can be had on eBay for pretty cheap and it doesn’t look terrible. However, Ecobee has a few features based on their room sensors that brought me back to it. Specifically, my favorite feature is the ability when creating a schedule to tell the Ecobee “only focus on the temperature/room sensors in these rooms” when determining the temperature of the house. So in the evenings/at night, I only care about the bedrooms. But during the day, I don’t care as much about the bedrooms bu worry more about the first floor/office. I’m sure there was a way to create a similar functionality with Zwave and other temperature sensors but having it all integrated/automatic in Ecobee was too nice a benefit to pass up.

Same goes for Ecobee being able to track things like HVAC maintenance, replacing the air filters, high/low humidity, etc. All could be done in other ways, but it’s nice that they are just built into a good looking device with a functional app. So I picked up an Ecobee Enhanced Thermostat which does not have any voice control (absolutely fine by me, I kinda hated that in the Ecobee3) and it all works.

My only gripe at the moment is that in HomeAssistant there’s a way to use Homekit to control the Ecobee so all the controls are local. Very handy. And in the old house I used that to pull the temperature/room sensors into HomeAssistant as it avoided any issues with cloud outages. However, I have a bunch of he “old” room sensors and those do not seem to play nice with HomeKit and Ecobee. They just show up as “unavailable.” Hoping that gets fixed soon.

Smart Home Hub

I don’t have much to add here beyond what I said in my last post. I remain a Google Home house/user. This year HomeAssitant is focusing on voice (in fact, it’s the Year of the Voice). I will be eager to see what they come up with and improve. However, I find I rarely need to use voice control as most things are handled by automations so I don’t know if I’ll switch any time soon.

The main thing I enjoy using the Google hubs for is the ones that have screens. Being able to cast things like my front doorbell camera video stream or weather/radar map when I wake up is super convenient. CATT is your friend here.

Zwave

I’m still a big fan of Zwave devices, though I will admit that it’s role in my house is greatly reduced. As part of my upgrade, I switched from the Aeotec 5th gen Zwave control stick to a Zooz 700 series stick. There is an issue with the 700 series sticks requiring a firmware update, you can read more about that [here](https://github.com/kpine/zwave-js-server-docker/wiki/700-series-Controller-Firmware-Updates-(Linux).

Really what made me stick with Zwave was Zooz’s switches being only $20-30. I really liked the ones I had in the old house (I left them there because I just didn’t feel like rewiring those AND the new house) so decided to stick with what worked. As you can see in the main image above, the vast majority of the switches in my house are Zen71’s and Zen 72’s. So far, no major issues. They work well, they have a lot of handy configuration parameters, and if I do find I want some Zwave sensors I have a strong mesh network.

I also got some Leviton ZW4SF fan speed control switches for the bedrooms. I have not had a chance to use these much yet as by the time I moved in winter was coming, but from my quick testing they work great. I’m really looking forward to being able to control fan speeds at the switch instead of with a pull chain.

Two other sets of devices that are new and worth mentioning. First is the Zooz Zen34, a battery powered paddle switch. I LOVE these things. Despite having the full floor/electrical plans for my house for months AND meeting with the electrician before install, I failed to grasp that the placement of the switches in my kitchen…well….sucks. Basically there’s 3 sets of lights: island lights, kitchen surround, and breakfast nook. Only the kitchen island has a 3-way switch and is controllable from the entrances to the room. And the other two light switches are placed in inconvenient locations. So to fix the problem, the Zooz paddle buttons let me add some additional “switches” to control the other lights (via Zwave group association) in the kitchen remotely. They aren’t a completely perfect solution as you can tell they aren’t real switches, but they solve the problem well enough I can overlook that failing.

The second device is the Ring Alarm Panel (2nd gen). I haven’t a chance to play around with this a ton, but it’s a Zwave alarm panel. It looks nice, is battery powered, and can do a whole host of things which will allow me to control the alarm system without a tablet. I probably will post some more details later, but for now here’s a good GitHub write up and another blog post with how to get it to play nicely in HomeAssistant.

In terms of sensors/other gear, I didn’t keep a ton from the old house as I really wanted to try to standardize as much as possible for the sake of consistency. So I sold all my old contact/motion sensors and went with Zigbee ones (more below). What I did keep was some flood/leak sensors and my Schlage door lock. I even picked up another Schlage lock (which was not easy to find even on Ebay) for my side garage door. I looked at other options like Ultraloq but there just seemed to be a bunch of problems, most notably that the Zwave lock just….doesn’t do anything smart. IMO really the only lock game in town is Schlage, and I’m hoping that the issues with stock are related to production issues and not Schlage getting out of the game.

Zigbee

While Zwave saw a major drop in usage, Zigbee now plays a significantly increase role in my house. That’s not to say I particularly like Zigbee over Zwave. In a perfect world where money was no object, I’d only have Zwave. However, there’s just so much more variety in the Zigbee world with some clever devices and just…well…they’re cheaper too. For now, I’m sticking with my “run both, pick whatever device is cheapest and best for the job” strategy, and I haven’t been disappointed yet. I really do think there are pros to running a blended network and very little downside.

For my Zigbee controller, for a long time I’ve been using the Deconz Conbee II and had very few issues, other than the Deconz software which I fixed by moving to Zigbee2MQTT. As it’s replacement, I picked up the Sonoff Zigbee 3.0 USB Dongle Plus-E. This is different from the popular -P model as it uses a different chip, the EZSP. You can see more of the differences on the Sonoff website. I decided to go with the E model because the EZSP chipset is the “future” of Zigbee as it will support Matter/Thread. The HomeAssistant SkyConnect uses the same chipset. Unfortunately the EZSP chipset is still pretty early in it’s support in things like Zigbeee2MQTT and I’ve run into a few issues along the way. It does seem to be getting better though. Haven’t had any devices drop off my network in over a week! If I was to go back and do it again I probably would have gotten the -P model of the Sonoff stick, but in the long run I’m sure the -E will be fine.

For sensors, I went basically all Xiaomi Aqara. I already had a number of the motion sensors and picked up some more, along with contact sensors. The contact sensors are hands down the best, as demonstrated by Rob over at The Hook Up. The motion sensors are….fine. I wanted to get the newer Aqara P1 motion sensors as they have the ability to change the detection sensitivity and cooldown. But I got one to test and it had horrible detection range. Instead I bough the older model from Cloudfree.shop.

I also have been playing with the Aqara FP1, which is a mmWave presence detection device. It’s actually good at detecting presence because unlike IR (used in most motion sensors) which requires a ton of movement, the FP1 can pick up on a tiny amount of movement and detect that someone is still in the room. It’s not 100% accurate (curtains fluttering from AC running can trip it, for example) but it’s miles better than I had before. Currently I’m testing them in my home office and family room. With some further tweaking/placement work I’m sure I’ll get it to a good spot. Note that Aqara at CES announced the FP2. This device won’t run on ZIgbee but rather WiFi/Matter, so that’s a pro to some and a con to others. Also note that you can’t get the FP1 in the US directly. Cloudfree has it available as does a few stores on AliExpress in China. I bought mine on AliExpress and had no issues.

I had the Ikea Fyrtur blinds in the master bedroom in our old house and liked them well enough. They’re a basic roller blind so they didn’t completely black out the room. I was fully prepared to buy sets for all the bedrooms in the new house when I came across the new Ikea Tredansen cellular blinds. I gotta say I’m in love with these things. They look like normal cellular blinds that you’d get from Lowes or HomeDepot and are full black out shades. All the bedrooms have them (and I repurposed the Fyrtur’s in the office). They open after we wake up automatically, close at sunset (or around 7pm if it’s in the summer), and will close earlier if it’s going to be a warm day (front bedrooms get a ton of direct sunlight). Yes they’re expensive but such a wonderful product to have. As an FYI the curtains do not come with screws (really annoying IMO). I used #8 wood screws (I want to say 1.5 inches long but I can’t remember exactly) and they worked great.

Finally, while as I said I really like the Zooz Zwave switches, I knew because I was going to be deploying a ton of Zigbee devices I needed a good Zigbee mesh, which means powered devices. I had some Sonoff Zigbee plug in outlets that worked as repeaters but again, I knew I needed more. Thankfully Inovelli had just announced their new Blue line of switches and I ordered a 10 pack. And I gotta give it to Inovelli, they made a really nice switch. While there was a defect in about 50% of the first production batch (only affected 2 of my 10), Inovelli was quick to get replacements out. I love the LED strip along the side (which I plan to use with automations for things like showing if a door is locked, for example) and rewiring a 3 or a 4 way setup was far easier as I didn’t have to touch the other switches (for now). I do plan to eventually swap the secondary switches for the Inovelli Whites. Inovelli also is working on some cool new stuff, like switches with built in mmWave detection and also a Blue series fan switch. Ultimately while I paid a bit more for the Blue switches, it was worth it in my mind.

Sound Machines

Ok this one hurts a little bit. I had this grand plan to use Volumio on a couple of Wyse 3040’s powered by POE, similar to what I had in the old house with some Raspberry Pis and battery backup. It was going to be this whole great thing that was locally controlled and bulletproof as it had worked flawlessly for years.

Then my son spent a week at his grandparents where my nephew has one of those “ok to wake” clocks that has an actual clock on it. And apparently my son learned what “7:00” was and that that was an OK time to wake up and get out of bed (instead of various times between 5 and 7am despite my “red light” strategy in the old house). So I began looking at various options. Ultimately I decided to go with the Hatch Rest+ (1st gen) which were on sale for the holidays.

As you can see, it has a built in clock, LED light, and sound machine, and it’s controllable over WiFi, not Bluetooth as the older models were. AND it has a battery backup that can last a few hours in the event of a power outage. So while I could have built something custom again with Raspberry Pis, some LEDs, and all that, ultimately it just seemed easier to bite the bullet and pay for a polished product.

What really sealed the deal is that there’s a HACS integration for HomeAssistant. When I ordered the Rests, only the Gen 1s were supported, but recently the 2nd gen Rests are supported. I don’t think any of the second gen features are all that compelling (other than maybe toddler lock but my solution there is just to put the clock up on a bookshelf that can’t be reached) so I didn’t mind saving some money. Every now and then I have some weird issue where the integration doesn’t work and devices don’t turn on when I have automations set to do it, but I’m chalking that up to my internet (not the most stable). And I found a workaround where I just reload the integration about 15-20 minutes before my kids wake up, go down for naps, or go to bed.

Doorbell/Cameras

I was going to talk about the new doorbell and camera setup, but given how long this post already is I’ll put this topic off to another time.

Other/Misc.

Just a few other miscellaneous things that I picked up that are now smart.

For home audio, I was tolerant of using the Google Homes as Chromecast devices for sound, but after a while that started to get glitchy. The obvious big name here would be Sonos, but I 1) hate how expensive they are and 2) feel like it’s yet another closed ecosystem. I know Ikea has their line of speakers but I saw them in person and they looked cheap. Instead I went with some Denon HEOS speakers (the 150 to be precise). I got 2 refurbished on eBay for pretty cheap. Haven’t played with them a ton, but sound wise they’re stellar. And I can use them in HomeAssistant, which I’m still exploring. I did consider doing in-ceiling speakers all running back to a central amp/receiver but given how little I listen to music it felt like overkill.

Naturally, just after the house went under contract, our washing machine started to make a weird sound and not spinning enough to dry our clothes. Le sigh, it was only 5 years old. Thankfully we’d excluded it from the purchase so we didn’t end up needing to buy a new one for the old house AND for the new house, but we still needed a new washer. And while smartifying it wasn’t the primary goal, it was a nice benefit. We went with the GE GFW850 and have been very happy with it so far. There’s a custom HACS component so I can get notified when the washer is almost done/done, and have an automation set to constantly nag me until the door is opened and wash moved.

The other appliances we chose to make smart was the dishwasher. With kids, we end up running the dishwasher a lot, and the worst thing with our old dishwasher was that we’d set it on a delayed cycle. That way we could add dishes until after we went to bed. The only problem was that if you opened the door, you had to press start again, otherwise the delay timer wouldn’t start. Nothing worse than coming downstairs in the morning to dishes that aren’t clean, especially if they’re needed for school/daycare. Enter the Bosch 800 series dishwasher. Now with a Aqara button next to the dishwasher, I can press the button and after we go to sleep the washer will turn on and run. Really handy.

I had been playing around with the idea of making some of the recessed lights in my house smart (as in the bulbs themselves) for color/ambience. I tested a lot of options, including some Zigbee options. Ultimately I decided I didn’t want to go down the smart bulb route as they were pretty expensive (~$30 a bulb, or more if you look at Hue). That’s when I found the Phillips Wiz bulbs. These are WiFi based bulbs, BUT they can be 100% locally controlled. I still didn’t want them in my recessed lights for a number of reasons (such as not having the lights all turn on in sync) but for my kids bedrooms they are perfect. I got the 100w ones so they’re nice and bright, and having color control is super handy. Favorite usecase is at night if their door is opened the light turns on red, which makes it easier to see without destroying night vision.

Lastly, despite my best efforts I now am a “proud” owner of a MyQ garage door opener. It literally was the only opener provided by the builder. And man I hate it. It took me probably 20 tries to get the damn thing connected to my WiFi network (using multiple devices even). The good news is that while now it’s on my network and it’s hooked up to HomeAssistant, I don’t use it. I have a more detailed post planned on this but basically there’s a Zwave relay and a dumb garage door button involved and I have complete local control of my garage door. F U MyQ.

Conclusion

OK wow this ended up longer than I was originally thinking. Lots of fun new toys, LOTS of automating still to do. Happy to answer any questions if you have them!

And just for lulz, here’s all the dumb switches I’ve taken out.

New Comment System - Giscus

Thu, 29 Dec 2022 21:30:00 +0000

Short post. Just wanted to announce that the blog now has comments thanks to Giscus. I was inspired by Alex/IronicBadger talking about it in his blog and also on the Self Hosted podcast. I always wanted a comment system but did not want to use Disqus for a variety of reasons. Alex’s lastest discovery was just icing on the cake.

Anyway, Giscus uses Github Discussions to create comment threads on posts. Since I already used Github for this blog, it was relatively easy to get set up. So now there’s comments! Yay!

Also, looking at how few articles I wrote this year I feel kinda sad. But I was waiting to write a lot of things because I moved this year. I wasn’t really do anything new for the past few months as I was in a holding pattern until I got into the new place. Now that I’m here and getting settled, I’ll make some time to document the new toys and things I’ve got set up. Expect more soon!

Revisiting BlueIris, Deepstack and Homeassistant

Sat, 30 Jul 2022 12:00:00 +0000

This is an updated post to my previous one on BlueIris and Deepstack. The setup I described in that post worked for over two years. However, I am in the process of moving and was looking to streamline things. And also seeing if there was any way to improve my already functioning setup. Back 2 years ago I had searched and searched for a way to get alert images from BlueIris to HomeAssistant but couldn’t figured out a way. Last week, I decided to try another search of the interwebs to see if anybody had figured it out, and I stumbled across this thread here. I fully recommend giving it a full read, though note that there have been changes since the initial post. I’ll also point out a few things below that might help streamline things.

This guide assumes a few things. First, you have BlueIris setup. Second, you have configured BlueIris to use MQTT to communicate to HomeAssistant. Though the dev is not working on the integration anymore, I’ve found this one to work the best. The dev also has an excellent guide on how to configure MQTT that’s worth a read. Note you don’t NEED to use the HASS integration; you can configure most of what is discussed below just using the MQTT setting described in the documentation for the integration. I have just found the integration to be easier to use.

Hardware

For the past 2 years, I’ve been running BlueIris on a dedicated box, specifically an HP EliteDesk G1 Mini, with an Intel i5-4590T and 8gb of RAM. Overall it did….fine. It could handle the cameras I was throwing at it (mostly Eufy indoor 2ks and a few Reolink 410’s), but whenever I would try to remote into it and open the BlueIris console, I would start hitting 100% CPU utilization and things were….slow. That’s partly why I felt the need to separate Deepstack away from my BlueIris box because I was afraid if I started trying to run Deepstack on the i5-4590T things wouldn’t go well.

As mentioned above, I’m moving into a new house (literally, it’s a new construction) which means I got to make some design decisions, including running conduit to many locations. Which opens up the possibility of outdoor POE cameras and hardwired indoor cameras. No more WiFi! I realized though that there was no way the EliteDesk was going to be able to handle the significant increase in megapixels I was about to throw at it. The current setup is 24 megapixels; the new house is probably going to be closer to 50 when I’m done. I looked into buying an upgraded EliteDesk as I love the small form factor devices, perhaps a 6th gen i5-6500T. The cost of one of those was around $200 on eBay and I figured for ~$100 more I could build myself a 10th gen, i5-10400 rig to match the hardware I have in my Proxmox server. That way I have some hardware redundancy in case of failures. I ended up going a bit over budget because I bought a new case and power supply but all in all I’m happy with my decision.

Here’s my hardware list:

Type	Item
CPU	Intel Core i5-10400 2.9 GHz 6-Core Processor
Motherboard	MSI B560M PRO-VDH Micro ATX LGA1200 Motherboard
Memory	TEAMGROUP T-FORCE VULCAN Z 16 GB (2 x 8 GB) DDR4-3200 CL16 Memory
Case	Fractal Design Meshify 2 Compact ATX Mid Tower Case
Power Supply	EVGA SuperNOVA GT 650 W 80+ Gold Certified Fully Modular ATX Power Supply

On a side note, I’m a huge fan of Fractal Design cases (literally every single case I’ve ever bought is a Fractal) and the Meshify 2 Compact doesn’t disappoint. Easy to build in, looks great.

After getting everything running, the CPU utilization now hovers around 5-10%, versus on the old hardware it was usually around 40-50%. And best of all, the power utilization only went up maybe 20 watts overall. That also includes the fact I added a spare 6TB HDD in the case so now I’m able to record directly to disk. Before I was shifting archived footage off to my NAS.

Intel gvt-g

On another brief sidenote, my first thought was to try to run Proxmox on the new setup to go along with the Proxmox server I already ran. That way I could possibly migrate things over in the case of needed downtime or hardware failures. I set it up, got the Windows VM running and successfully passed through the iGPU to the VM. I started up BlueIris and everything seemed to be working. For about 24 hours I had no issues and I thought it was going to work. Then suddenly out of the blue I started getting system lockups on both the VM and the Proxmox host.

Ultimately, I do not recommend trying BlueIris in a VM and try to pass through the iGPU for QuickSync. You can read more about the issues people have had with gvt-g here. My experience with performance wasn’t as bad as IronicBadger’s was, but the system instability was a deal breaker.

Deepstack Setup

Initial Configuration

As I described in my previous post on Deepstack and BlueIris, what I was after was a way to get images of persons/cars being detected to my phone via HomeAssistant. The other ways I had tried didn’t allow me to filter out the notifications so that I didn’t get alerts when I really didn’t care to get them (like when I’m leaving home). So I worked out a somewhat convoluted approach in which:

BlueIris would detect motion and trigger an alert.
HomeAssistant via MQTT would pick up on that trigger and in turn trigger an image scan with Deepstack using a custom component.
If Deepstack determined there was in fact a person/car, it would then alert me.

While it was overly complicated, it did in fact work pretty well. I reliably got alerts via Telegram whenever a person or car was detected. I think my main problem with this approach was that there was some lag. First the alert would have to go out via MQTT to HASS, then HASS would have to take a snapshot and send it to Deepstack, which then had to process it and return it to HASS, which then would send my the notification. I’d say the overall time between motion event and alert was 5-8 seconds. Not bad, but not great.

Now that I finally had better hardware that I felt more comfortable running Deepstack and BlueIris at the same time, I started to play around. Hat tip to Rob over at TheHookup on Youtube for a pretty detailed explanation of how to setup BlueIris and Deepstack. I suggest giving this video a watch:

This one is older (and somewhat outdated) but I think still worth a watch:

I haven’t yet tried the substream recording/upgrading to higher res on motion because 1) HDDs are cheap and 2) the Eufy cams I’m running don’t have substreams so it doesn’t seem worth it. Maybe with the new cams (I’ll be sharing which ones I’m going with later) I’ll start doing this.

Alert Settings

I won’t go through all the setup for Deepstack and BlueIris as I think that’s well covered in the video. I also think Rob’s suggested settings for AI are pretty decent, though I turn off “Begin analysis with motion-leading image” to try to get the best snapshot of the person or car I can. Here’s my typical settings:

You absolutely should tweak the settings and the confidence levels to fit your own needs. Now to get the alert image into HASS, you need to go to the Alerts tab. At the bottom, click “On alert…” Click the plus icon and click “Web request or MQTT”.

For MQTT topic, I would recommend something like BlueIris/image/&CAM where &CAM will automatically be substituted by BlueIris to be the shorthand name for your camera (makes copying/pasting for multiple cameras easy). I like to make sure all the images are under the same subtopic just for organizational purposes, but you could configure yours however you like. The Post/payload must be &ALERT_JPEG as that’s the secret sauce that tells BlueIris to send the alert image to MQTT in Base64 encoding. I also enabled MQTT retain message so the camera entity in HASS you’ll set up below always has an image to pull and you shouldn’t get any errors. Here’s an example of what mine looks like:

You also can modify what profiles should the alert image get sent. For example, I have a “Vacation/Alarm” profile where all my cameras (indoor and outdoor) send alerts, but a “Default” profile where indoor alert images are not processed. Nobody wants pictures of them walking around in their underwear IMO….

Also, I highly suggest ordering these alerts so that the image is sent first. If the alert triggers the motion sensor in HASS first you may not get the updated image.

HomeAssistant Integration

Now that the hard part is done, the HASS integration is actually really simple. All you really need to do is create an MQTT camera based on the topic you set above. And specifically use the b64 encoding option. Also, remember that starting with 2022.09, all MQTT entities need to be under the mqtt key in your configuration.yaml file. So your configuration may look something like this:

mqtt:
	camera:
		- topic: BlueIris/image/driveway
		  name: Alert Driveway
		  unique_id: driveway_alert
		  encoding: b64

Change the topic to match whatever you configured in BlueIris. Restart HomeAssistant and you should be able to see a new camera entity showing the last alert image from BlueIris! Now to send the image I’ve found the easiest way is to take a snapshot to my local disk and then send that image file to my notification app of choice, which is Telegram. You could pick whatever you prefer without much issue.

I’m able to do a little more fine tuning on the HomeAssistant end, like rate-limiting the number of notifications I get to 1 ever few minutes (so my phone doesn’t melt from all the notifications). Now my notification setup looks like this:

BlueIris would detect motion and triggers a scan with Deepstack
If Deepstack determines there’s a person, the alert image is sent to MQTT and the motion sensor in HASS is triggered.
HASS then notifies me.

The processing time here is more around 3 seconds (instead of the 5-8 before). It’s not a huge improvement but it feels far less convoluted.

Conclusion

I’m pleased with both the new hardware (it’s so much faster!) and the new way to get my alert images into HomeAssistant. There wasn’t anything “wrong” with my previous setup as I don’t think the performance/speed improvement is all that great, but it does feel a lot cleaner and simpler than before. Also feels easier to maintain should there be issues in the future.

Deconz v. Zigbee2MQTT

Tue, 24 May 2022 22:30:00 +0000

I’m a big believer in both Zwave and Zigbee. Both have their pros and cons. Zwave in my opinion is a better technology because of the lack of interference and it propagates further/wider due to being a lower frequency. However, the devices are more expensive and there’s less overall variety. Zigbee on the other hand shares the same frequencies as Wifi (and other technologies) in the 2.4ghz band so interference can be an issue. However, the devices are usually significantly cheaper and there’s a lot more variety when it comes to devices.

So instead of picking one and sticking to it, I run both in my house to play to the general strengths of both. When it comes to Zigbee, I’m a big fan of the Xiaomi Aqara line of devices. They’re cheap, they work great, and there’s a lot of nice devices. My personally favorite are these button switches that I use all over the house to trigger various automations.

For a hub, I’ve long used a Conbee II. There are a number of hubs out there but I’ve found the Conbee to be the best. The Conbee is manufactured by a company called Phoscon/Dresden-Elektronik, who in turn make a controller software called deConz (Note: I may have the ownership part wrong, I tried researching but couldn’t find a definitive answer). When I first started out, I figured why not use the software that’s made by the company that makes the hardware. There’s probably good synergy there. And I was right. Sorta.

deConz

DeConz as software works fine. It is able to interface with a good variety of devices and it controls them just fine. What was not so wonderful was 1) pairings could be an absolute nightmare and 2) occasionally devices would drop off the network and require a repair. And third, unless you had lights in your network in a group, EVERY TIME you logged in you’d have to click through 3 screens about setting up light groups. Was a real PITA and I could never figure out how to turn that off. The first two issues though were real pain points. For example, pairing my Ikea Fyrtur curtains took literally days and dozens of attempts to get paired. And even then occasionally they would just stop responding for no apparent reason.

I also found it can sometimes take a while for deConz to support new devices. They do seem to want to take their time and get it right, but if you want the latest and greatest devices it can be frustrating at times. And while it’s technically possible to run Over the Air updates on deConz, I struggled to get it to work and just gave up.

In terms of integrating with HomeAssistant, it was really quite simple. There’s an official integration, devices just show up, and it works fine. To get the button events I had to use an even listener but that wasn’t a major problem. I didn’t have an issue with this part of using deConz.

Zigbee2MQTT

I’m not really going to go into the “how to set up Zigbee2MQTT” because I think 1) it’s really pretty straightforward and 2) there are a lot of other posts/videos on how to do it. So instead, I’m going to be focusing on my thoughts/impressions.

I wanted to test out Zigbee2MQTT as an alternative to deConz for a while, particularly because I’d heard great things about the sheer number of devices it supported, including the new Aqara P1 motion sensor I picked up off Amazon. One evening, I decided I would give it a try. And stupid me, I thought I could just stop deConz, start up Zigbee2MQTT, add the P1 sensor, try it all out and then go back to deConz without any issues.

That….did not go according to plan. I was able to set up Zigbee2MQTT just fine, got it all connected to the MQTT broker and everything worked fine. Was done for the night, shut down Zigbee2MQTT, restarted deConz and started wrapping up for the evening. When I noticed some of the motion sensors didn’t seem to be working because the lights weren’t turning on. Went into deConz and….none of the devices were showing up. Uh oh….

Decided to bite the bullet since I was going to have to repair everything anyway and just switch everything over to Zigbee2MQTT. That evening I got the bare essentials repaired and then got the rest repaired the following day, along with renaming things so my existing automations would continue to work. It was painful with about 25 devices but not horribly so. The biggest thing that made it easier was the pairing was so much easier with Zigbee2MQTT than Deconz. For starters, everything paired without much issue. Including the Fyrtur blinds that I struggled so mightily with when using deConz. Second, Zigbee2MQTT opens up pairing for 5 minutes and allows you to pair as many devices as you want/can in that time period. deConz was one at a time, and like with the entry screens, the UI was cumbersome to navigate.

The devices just showed up for me in HomeAssistant under the MQTT integration. I do wish they were separated from some of my other MQTT-based devices, but they were easy to configure/rename so it’s a minor quibble. And doing OTA updates via Zigbee2MQTT is a breeze. Not many of my devices actually had an update to run

Conclusion

After a few days, I’m very happy I switched. Devices that were giving me trouble with deConz are working just fine now with Zigbee2MQTT, with one issue regarding the humidity sensor I use in my shower along with a trend binary sensor to detect whether someone is showering. I’m sure I’ll figure it out.

In terms of “should I switch off deConz to Zigbee2MQTT” the answer is it depends. I was kinda forced to repair everything so I figured why not make the switch, and my network is relatively small at 25 devices. If deConz is working fine for you and you aren’t using unsupported devices, I don’t think there’s a super compelling reason to switch. But if you are just starting out with Zigbee or find yourself wanting/needing to switch, then definitely give Zigbee2MQTT a go!

Valetudo on a Dreametech Vacuum

Mon, 25 Apr 2022 23:00:00 +0000

For a while I’ve been using a trio of Eufy 30C Robovacuums to clean my house. And they worked….fine. They were your standard bump-and-turn variety of vacuum robot, meaning no floor mapping/LIDAR or other advanced features. And like I said, they worked fine. Until they didn’t. I’ve already had to replace one under warranty and I had another die about 6 months after the warranty ended. I thought about replacing it with another Eufy but given that I’ve gone through two in less than two years I wasn’t keen to drop another $200+ without exploring the alternatives.

I had been curious about Valetudo for a while. It’s open source software that, in short, removes the cloud from your robot vacuum. Meaning it’s entirely locally controlled. Think of it as Tasmota for robot vacuums. However, Valetudo only works on robots in the Xiaomi ecosystem. Roborock used to be the go-to, but they’ve apparently patched things in such a way that it’s not easy to root anymore. They still make great vacuums, so I almost went with them, but instead I started looking at DreameTech.

Hardware

DreameTech uses the same underlying tech as Xiaomi/Roborock. They make a number of robot vacuums. The specific ones I have/looked at were the Dreametech D9 and Z10 Pro. There also is the L10 Pro but I did not get that one. The main difference as best I can tell between the L10 and D9 is that there’s more memory, greater suction, and longer run time. To me that was not worth the extra $200.

Overall the Dreametech vacuums are leagues better than Eufy. For starters, the LIDAR mapping works great. I’m able to control which rooms the vacuum cleans and I know it will clean the whole room instead of hoping that every spot gets covered with many random passes. I also can add “no go zones” where I don’t want the robot to go. For example there’s tassles on the carpet or an area where I know the robot will get stuck. Also as a result of the mapping, the Dreametech vacuums are also a lot more efficient. The Eufy vacs used to run for ~1 hour. The Dreametechs run for closer to 40 minutes.

D9

The D9 is a great, basic vacuum. It comes with LIDAR for room mapping, which works really well as discussed above. It’s really a no-frills vacuum that you can get for ~$250 on Amazon. It does come with a mop attachment but…it’s pretty lackluster. Basically it just drags a wet cloth around your floor. It can do an OK job but it won’t work any miracles. My biggest knock on it is that it sometimes gets twitchy with Valetudo and loses the map data or drops the wifi connection. But I think for the price, it’s great.

Z10

The Z10 is basically the same robot as the D9 with a few more bells and whistles. The biggest extra features are the self-emptying bin and the object avoidance. Honestly not sure that was worth the premium but… #forscience. Both features are nice to have (and the self-emptying part does save me some time) but neither are revolutionary game changers.

Installation

Installing Valetudo on the Dreametech vacuums is both simple and tricky. The first thing to know is that the most commonly used method to root the Dreametech vacuums was patched in September 2021. However, from my experience no robot has yet shipped with the newer firmware. For example, my Z10 Pro was produced in December 2021 and I had no issues rooting it. This may change in the coming months so be aware. Newer root methods may be available, open up a ticket with the Dustbuilder dev here.

Oh, that’s one important thing to understand here: Valetudo is the software you use to control the robot. To root the robot, you use Dustbuilder. It took me a combination of the Valetudo instructions here and the Dustbuilder FAQ here to figure the whole process out. Below are some of my tips/experiences to help along the way.

Also keep in mind you will need a UART adapter to flash the robot. They’re pretty cheap on Amazon. And also you’ll need a wifi device later, preferably one that also runs Linux. I used a Pi for the job.

EDIT 7/15/2022: Valetudo has revised and vastly improved the rooting instructions here. I’ll keep the below for posterity sake, but at this point I believe you can simply follow the Valetudo guide and not mine. Most notably, the introduction of the HTTP-Bridge helper avoids a lot of the pains I had.

Once you have what you need, I would suggest starting with the Valetudo instructions. You’ll need to pry open the robot (takes a decent amount of force but just take your time). Connect the pins to the headers and to your computer. Keep following the instructions until you get to the part where you need to login. To get the password you’ll need the FULL serial number (meaning BOTH lines in the dustbin. I used the calculator linked in the Valetudo docs. Now we’ll need to switch over to the Dustbuilder site to get the firmware we need to continue.

On the Dustbuilder site, pick your vacuum, provide a VALID email address (firmware will get emailed to you) and provide again the full serial number for the vacuum. I suggest keeping the Patch DNS and Preinstall nano/curl/wget/htop/hexdump options selected. And also select “Prepackage valetudo” obviously. Click “Create Job” and wait a few minutes for the firmware to be built. Download the firmware and move it to the device you’re using to root. In my case, the Raspberry Pi.

Now that you have the firmware, it’s time to start rooting. But first, BACKUP THE VACUUM! This will help you if you run into problems. From the shell with the robot, execute the following command: cd / ; tar cvf /tmp/backup.tar /mnt/private/ /mnt/misc/. This will create a tarball of the needed files.

Now comes the part that I found to be unclear/difficult and took me a while to work around. First, you’ll want to install Python’s webserver module. Run python3 -m pip install --user uploadserver to install. Next, you need to connect your computer/Pi to the vacuum via WiFi. I used raspi-config to find and connect to the correct SSID.

Once you’re connected, you’ll want to run the Python webserver to allow uploads. python3 -m uploadserver on the computer/Pi should be what you need. Then on the ROBOT, type curl -X POST http://<your-laptop-ip>>:8000/upload -F 'files=@/tmp/backup.tar'. Keep in mind, your-laptop-ip is probably going to be something like 192.168.5.2 or something like that. That took me longer than I care to admit to figure out. The backup tarball should now be on your Pi (double check with an ls if you’re paranoid like me).

You’ll need to do something similar to move the Dustbuilder firmware tarball over to the robot. I ran a Python webserver on the Pi in the directory where the firmware was located with python3 -m http.server. Then on the robot, change into the /tmp directory (cd /tmp) and run wget {url-of-firmware.tar.gz} where again the URL will be the IP of your computer/Pi along with the filename. So for example wget 192.168.5.2:8000/dustbuilder.tar.gz. The file should be transferred over. Next run tar -xzvf {name-of-firmware.tar.gz}.

Make sure the robot is docked/plugged in and then run ./install.sh. This will run the root script and get your Valetudo. As noted in the Valetudo docs, the robot will reboot after it’s done installing. BE PATIENT THIS CAN TAKE A WHILE. After the reboot, you’ll see the MOTD as described in the docs. Hooray, success!

YOU ARE NOT DONE YET.

Let me repeat that: YOU ARE NOT DONE YET. The robot has two system partitions, meaning you need to root the robot AGAIN in order to fully gain control. You’ll need to start back over at the step where you copied the Dustbuilder firmware over onto the robot and go from there again. You don’t need to backup the robot again. Done it a second time and the robot has rebooted again? Congrats, you’ve now rooted your new robot vacuum and own it without the cloud!

Conclusion

From here, you’ll need to connect the robot to your wifi. I suggest using your phone for this part. There is an Android app available for Valetudo (I’m no aware of one for iOS, sorry). But you shouldn’t need it. Just connect to the robot’s Wifi SSID and go to either 192.168.5.1 or 192.168.8.1. From there you should be directed to the Valetudo config page where you can provide the wifi credentials.

Now that you own the robot, you can easily get the robot to appear in HomeAssistant (just follow the Valetudo docs for this, and keep in mind you’ll need an MQTT broker). I also highly recommend the Valetudo map card. I played around with some customization where I can control what rooms gets cleaned on what day and how many passes to do but I’ll go into that in a later post.

Ioniq 5 - My First EV

Thu, 14 Apr 2022 22:32:00 +0000

Long before the current spike in gas prices, I was in search of a new car. Generally I try to drive cars until they die, but the Mrs.’s car was approaching 150,000 miles and 10 years old so we figured it was time for a new one. She really wants a truck for her next car, which we don’t need currently (and she wants an electric too!) so we figured we’d wait. That put me in search of a new car for myself.

I’ve loved my Hyundai Ioniq hybrid for years. Sure it’s no muscle car but it gets me from A to B, gets great gas mileage (45-50 in the winter, 55+ in the summer) and is pretty roomy. We started looking at another Ioniq hybrid when I saw the Ioniq 5, Hyundai’s new fully electric crossover.

At first, I was totally putoff by the retro look. But after some reconsideration, it ended up on the top of my wanted list. And eventually I finally got my hands on an Ioniq 5. Granted, I was more interested in an SEL trim than a Limited, but the Limited is what became available. And boy is it a sweet ride. It’s loaded with all kinds of fun tech (heads up display on the windshield, 360 camera, phone-as-a-key) and…yeah it’s all electric so no more gas.

As a brief aside, two absolutely invaluable sites that aided my search were TheEVFinder and a [user-submitted dataset that tracks dealer markup over MSRP](EV Dealer Markup Tracker). That combo eventually led me to my car.

Technology/Tools

Now having an EV is great, but it definitely comes with a few hiccups/requirements. Below are some of the tools I’ve come across that help EV owners plan trips and keep track of the status of your car.

HomeAssistant

First, there’s the all important HomeAssistant integration (naturally). This is a custom integration you can find here. Unfortunately it’s not feature complete (at least not in the US) so important things like controlling charge percentages aren’t yet working. Hopefully soon. There is a fork out there as well if you want to look at that.

Another big limitation on this integration is on Hyundai’s end. They limit the number of API calls that can be made to somewhere around 30/day in the US. So the integration only does a hard pull every 4 hours by default and does not update during the evening (can be customized). The lack of frequent updates does limit data resolution, particularly when it comes to charging the car. But have to live with it. As a result of this limitation I haven’t done a whole lot of automations yet. Mainly just tracking the battery level and sending an alert if the battery runs low, or on Sunday evenings if the battery isn’t charge enough to get me through the week.

That doesn’t mean I haven’t worked on creating a nice UI to view the car stats. I did borrow most of the template from another user in the HASS forum, with some tweaks. It does require a few custom cards: custom:stack-in-card and custom:bar-card. I’ll have the dashboard YAML at the bottom of the post. The other tweak I’ve done is creating a few sensors to track the car’s performance in miles per kilowatt hour. It takes a bunch of input_number entities and template sensors and I’m not really sure it’s worth the effort. If someone is really interested in how I did it, please feel free to reach out and contact me and I’m happy to share.

A Better Route Planner

While the charging infrastructure is OK, there are some definite gaps and going on a road trip is not nearly as easy as in a gas car. This next tool is absolutely invaluable when it comes to planning said road trip. A Better Route Planner (ABRP) is both an app and a website that, well, plans your route better. You type in your start point and destination, along with your EV model and some basic charge information, and it’ll map out a route for you, including where to charge, for how long, and to what percentage.

If you pay for Premium, you can take it one step further and have the app factor in real-time weather and traffic information, both of which can have a significant impact on an EV’s range. It also adds some other useful perks, like charger availability and Android Auto/Carplay functionality.

To go one step even further, you can pick up a Bluetooth Low Energy ODB2 plug that allows ABRP to read the actual remaining battery percentage. This allows ABRP to recalculate your actual consumption (measured in miles per kWh) and re-estimate your charging needs. There are a few models recommended by ABRP here. For the Ioniq, for some reason the Vgate one wouldn’t start back up when the car was off. I ended up going with the ODBLink CX which has worked great (though is on the pricier side). Note the big huge security warning with this though and make sure you’re OK with the ramifications.

PlugShare

Now this next one isn’t necessarily unique, but I’ve found PlugShare to be one of the better tools for figuring out where plugs are located. You can filter by all sorts of conditions and can see user reviews/check-ins to better determine if a charger is working or where it’s located. In combination with ABRP I have been able to map my trips with relative ease.

Conclusion

Recently I put all these tools to the test when I took a 700 mile road trip. Overall I was pretty pleased, though I will say ABRP was super conservative when it came to estimating my charging needs. For instance, on basically every leg of my trip it indicated when I started that I would arrive with ~10% charge. With one exception (first leg of the day when it was slightly chilly and I was going 70 MPH which was the speed limit), I arrived with way more left in the tank. Sometimes to the tune of 30% extra. While I’d rather it be conservative and have me charge more often to be safe, I did find this a bit annoying. Perhaps after some further tweaking and refinement it’ll improve. Still for now it’s the best tool I’ve found for the job.

Dashboard YAML:

type: vertical-stack
cards:
  - type: picture-entity
    image: /local/ioniq5.png
    entity: device_tracker.bluey_location
    name: Bluey
    tap_action:
      action: none
    hold_action:
      action: none
    card_mod:
      style: |
        ha-card {
          --ha-card-background: transparent;
          box-shadow: none;
        }
  - type: custom:stack-in-card
    mode: vertical
    keep:
      background: false
      border_radius: false
      box_shadow: false
      margin: false
      outer_padding: false
    cards:
      - type: glance
        entities:
          - entity: binary_sensor.bluey_plugged_in
            name: Charger
          - entity: binary_sensor.bluey_charging
            name: EV Battery
          - entity: sensor.bluey_range_by_ev
            name: Range
        show_state: true
        show_name: true
        state_color: true
      - type: custom:bar-card
        entities:
          - entity: sensor.bluey_ev_battery
            name: Percentage
            icon: mdi:car-electric-outline
            color: '#161616'
            severity:
              - from: '25'
                to: '100'
                color: '#006600'
              - from: '15'
                to: '25'
                color: '#c1c100'
              - from: '0'
                to: '15'
                color: '#660000'
          - entity: sensor.bluey_battery_capacity_remaining
            name: Capacity
            icon: mdi:battery-charging
            color: '#161616'
            max: 77.4
            severity:
              - color: '#006600'
                from: '50'
                to: '78'
              - color: '#c1c100'
                from: '25'
                to: '50'
              - color: '#660000'
                from: '0'
                to: '25'
      - type: glance
        entities:
          - entity: binary_sensor.bluey_tire_pressure_all
            name: Tire Pressure
          - entity: sensor.bluey_car_battery
            name: 12v Battery
          - entity: binary_sensor.bluey_engine
            name: Engine Status
        show_state: true
        show_name: true
        state_color: true
      - type: markdown
        content: ___
      - type: glance
        entities:
          - entity: sensor.bluey_mileage_since_last_charge
            name: Trip
          - entity: sensor.bluey_comsumption
            name: Consumption
          - entity: sensor.bluey_odometer
            name: Odometer
        show_state: true
        show_name: true
        state_color: true
      - type: custom:button-card
        entity: sensor.bluey_last_update
        show_last_changed: true
        styles:
          grid:
            - grid-template-areas: '"i n l"'
            - grid-template-rows: 100%
            - grid-template-columns: 25% 45% 30%
          label:
            - justify-self: start
          name:
            - justify-self: end
            - margin-right: 10%
        name: 'Last Updated:'
        aspect_ratio: 7/1
        tap_action:
          action: call-service
          service: kia_uvo.force_update

Controlling OPNsense Firewall Rules from HomeAssistant

Tue, 14 Dec 2021 10:20:00 +0000

I’ve been running OPNsense for a few months now and absolutely love it. It can do basically everything I want/need and does it with a relatively easy to understand interface (with some exceptions). It also has a good API. I talked more about OPNsense in a previous post so I’m not going to revisit a discussion again of OPNsense itself. But in this post, we’re going to talk about being able to remote turn on and off firewall rules…from HomeAssistant!

I try to minimize my firewall rules as much as I can, and especially port forward rules that expose my network to the outside internet. Thanks to the global pandemic, I’ve been working from home a few days a week. Therefore I find I generally don’t need to remotely access my network as much as before. However, every now and then I need to get in to fix something, so I leave SSH open (on a non-standard port and secured with SSH keys!) to a bastion server.

If I’m working from home though, I don’t need to leave open SSH (and would prefer not to even with all the security precautions in place). I started toggling the port forward rule on/off manually but then I forgot one day and needed to get back in and couldn’t, which was infuriating. So I started looking for a solution. Here’s how I did it.

OPNsense Setup

First thing is we need to make some changes on OPNsense. First, you need to install the os-firewall plugin. This plugin adds an “Automation” section to the Firewall menu in OPNsense. There are two options, “Filter” and “Source NAT.”

I’m going to assume you already have a port forwarding rule set up in NAT->Port Forward. You’ll still need this part set up. What’s going to change is that you will no longer need a rule under Rules->WAN We’re going to replace that with a rule under Automation->Filter that will be toggleable from the OPNsense API thanks to the os-firewall plugin.

Click the plus icon in the bottom right corner and fill out the information as you need, depending on what you’re trying to accomplish. Here’s my rule setup for allowing SSH:

It’s very similar to how you would set up a normal firewall rule in OPNsense. There are plenty of guides for how to manage/setup OPNsense firewall rules. Personally I found this site incredibly helpful.

Note, you could do this for virtually any firewall rule. It doesn’t necessarily need to be a port forward rule.

The next step is you need to create a REST API token. I went over this in my WOL blog post so I’ll direct you there for how to do it. You could even reuse the same API, just expand it’s permissions a bit if you wanted. The only difference between the WOL role and this one is that you need to add the right privileges. For the firewall, that’s GUI:Firewall: Rules: API.

The last thing we need is the UUID from the firewall rule we set up in automation. The easiest way to get this IMO is to go to System->Configuration->Backups and click “Download configuration.” Search for the name of your firewall rule and the UUID should be right there.

HomeAssistant

Now that everything is set up in OPNsense, let’s move to HomeAssistant. First off, I want to give a big thanks to this post for basically laying out exactly what the API calls needed to be. I just made modifications to make it work with HomeAssistant.

I’m not going to explain a whole lot because I think it’s all relatively self-explanatory, but here are the various bits and pieces you’ll need to get set up.

First is the REST Commands. These are what make the API calls. Substitute out the UUID from above and the IP address of your firewall. You’ll also want to set the username/password to be the keypair you created above. I store mine in secrets.yaml.

rest_command:
  ssh_rule_toggle:
  url: 'http://OPNSENSE_URL/api/firewall/filter/toggleRule/UUID_OF_AUTOMATION_RULE'
  method: POST
  username: !secret opnsense_user
  password: !secret opnsense_key

  ssh_rule_apply:
  url: 'http://OPNSENSE_URL/api/firewall/filter/apply'
  method: POST
  username: !secret opnsense_user
  password: !secret opnsense_key

Next is a binary sensor that will check to see if the firewall rule is enabled or disabled. Again, swap out your firewalls IP address and the relevant UUID.

binary_sensor:
  - platform: rest
    name: SSH Firewall Rule
    scan_interval: 30
    device_class: connectivity
    resource: http://OPNSENSE_URL/api/firewall/filter/getRule/UUID_OF_AUTOMATION_RULE
    username: !secret opnsense_user
    password: !secret opnsense_key
    value_template: "{{ value_json.rule.enabled == '0' }}"

Since you need two commands to enable/disable the firewall rule (one to toggle the rule on/off, the other to actually apply the change), I found it easier to create a simple script to call both commands.

script:
  remote_ssh_firewall:
    alias: Toggle Remote SSH
    sequence:
      - service: rest_command.ssh_rule_toggle
      - service: rest_command.ssh_rule_apply

And finally, a nice switch that you can both use in automations and also in the Lovelace UI to toggle the rule on/off.

switch:
  - platform: template
    switches:
      remote_ssh:
      friendly_name: "Remote SSH"
      value_template: "{{ is_state('binary_sensor.ssh_firewall_rule', 'on') }}"
      turn_on:
        service: script.remote_ssh_firewall
      turn_off:
	    service: script.remote_ssh_firewall
	  icon_template: mdi:wan

And here’s what the final result looks like:

Conclusion

There you have it. You can now automate enabling or disabling firewall rules in OPNsense as needed. There are probably other ways to do this, such as with a VPN, but this works for me. Hope someone finds it helpful!

NOTE: I updated this post in November 2023. Opnsense seems to have altered how rules are handled and I noticed that when the rule was “enabled” I was not able to get through the firewall, but if it was disabled I could get through. I swapped my setup around and everything is working. Not sure why or when the changed happened.

My Love/Hate Relationship with Drone.io and CI

Thu, 18 Nov 2021 08:38:00 +0000

Introduction

Let’s talk about Drone. No, not this kind of drone

This kind of Drone! The Continuous Integration kind. The one that helps you with automatically doing all kinds of fun coding things. Basically, Drone is a self-hosted alternative to something like GitHub Actions or Jenkins or Travis CI. Basically, if there’s something with your code that you want to run automatically after some action is taken, then CI is for you.

I’ve been playing around with Drone for about a week now, and while ultimately I’m pretty happy with it, I’m also incredibly frustrated by it. My initial foray into continuous integration projects started a few months ago with GitHub Actions so I’m entirely willing to admit that it may be user/beginner error. However, there are seemingly basic things that should just work but don’t. Other things that should be easily to set up but aren’t. The number of times I wanted to pull hair out over something not working is beyond count.

Deploying

Setting up Drone is really very easy. There are two basic parts: a server and a runner. There are a number of different runners available but I prefer Docker. It’s possible to run the Drone server on a different machine from the Drone runner. For instance, I run the server on my VPS, but the runner is on an machine at my house. Obviously my hardware at home has beefier specs that the VPS so builds go much faster. I’m not entirely sure how the runner and server communicate but it did not require me creating any firewall rules on the server or at home. And any kind of lag/delaying in triggering builds is hardly noticeable.

I think the documentation here is actually pretty well done and there is’nt much that needs explaining. Just follow the instructions and you should be up in no time. The main limitation I ran into is that you can only tie the Drone server to one of the source control platforms (GitHub, Gitea, Gogs, etc.). So if you have multiple places you use, you’ll need multiple instances of Drone. Given Drone’s resource usage is basically non-existent, that shouldn’t be an issue.

As an FYI, to get your account token which you’ll need for the API, just navigate to https://YOUR_DRONE_SERVER/account.

Examples

Things I’m doing with Drone:

Running a yamlllint and where appropriate ansiblelint on every commit;
Building various docker containers and pushing to Docker Hub and GHCR;
Running a config check on my HomeAssistant configuration files on every major HomeAssistant release (including the beta releases) so I can know ahead of time if my configuration needs fixing before I deploy the update;
Building and deploying this blog, along with a staging site I can use for testing/debugging purposes.

In reality, the sky’s the limit with automating like this. One of the more difficult setups I achieved was automatically building a modified Docker container of Nextcloud that includes Samba and imagick. I prefer the official Nextcloud image but it doesn’t ship with those packages installed so I needed to create it myself. I didn’t want to have to manually trigger the build every time, so now, Drone runs a check script every 6 hours and using one of my new favorite tools, skopeo, checks to see if there’s an updated Docker image and if there is, update the Dockerfile with an updated tag and tag a new release on GitHub. That then triggers a new build of the docker image, which is pushed to DockerHub and GHCR.io, and then the images are deleted. Again, that’s just one example (and I’d even say a fairly basic one) of what Drone can do.

Likes/Dislikes

Likes

Let’s start off with the good, as there’s a lot of good things with Drone that I liked. As you’ll notice though, some things that I liked I also ran into aspects that I disliked.

Use Any Docker Image

Probably the handiest feature is you can literally use any available Docker image/container to complete your task. Drone has a number of plugins available (frustratingly the page doesn’t support HTTPS for some reason….) but I found that in the end it was often easier to just build/do it all myself. Sometimes that meant a lot of trial and error, and teaching myself more bash scripting like nested if/else statements. But regardless of how you do it, the point is you probably can do it with Drone given the flexibility what container you use.

More Flexibility

Drone lets you use bash to your hearts content. So you can use the commands portion of your pipeline to run basically any command you want. Including referencing bash scripts you may have in your git repo. Note however that if you cannot use the commands option with a Drone plugin. As soon as you execute a command the plugin basically becomes a normal Docker container with no other special instructions.

Triggering Builds via API

As discussed below, I ran into some limitations when it came to triggering or limiting my builds. Using the API (and of course, using HomeAssistant to set off the trigger, thus automating my automations) was very handy. Triggering a build is pretty easy:

curl \
-i "https://drone.example.com/api/repos/USER/REPO/builds" \
-H "Authorization: Bearer YOUR_TOKEN"

To add variable, add a ?YOUR_VAR=test after /builds like this:

curl \
-i "https://drone.example.com/api/repos/USER/REPO/builds?VAR=test" \
-H "Authorization: Bearer YOUR_TOKEN"

Dislikes

That’s not to say that Drone is perfect. I really struggled with some things that in my opinion should be really straightforward (or better documented).

Cron

Let’s start with an easy one. Drone includes the ability to trigger builds based on a cron schedule from the GUI.

Notice, however, that there’s no way to create a custom schedule beyond the defaults. Also, you can’t set the time the cron triggers. By default it’s midnight UTC. Oh, and you can’t set the timezone from the GUI either…I had to mount /etc/localtime as a volume for it to set it to my local timezone.

You can create a custom schedule, however it must be done via the API. Which isn’t hard, but it’s an annoying limitation that just shouldn’t exist. Here’s an example call you can use:

curl -X POST https://drone.example.com/api/repos/USER/REPO/cron \
-H "Authorization: Bearer YOUR_DRONE_TOKEN" \
-H "Content-Type: application/json" \
--data '{"name": "EXAMPLE", "expr": "* 0 */6 * * *", "branch": "main" }'

Note the cron scheduling includes seconds as an option so it’s 6 asterisks instead of the usual 5. So seconds, minutes, hours, day of month, month, and day of week. Once it’s created, you can see the schedule along with when the next execution will be.

Triggering Builds

Something that I ran into was some limitations when it came to both conditions and triggers. The most frustrating limitation is that while you can have multiple conditions, ALL conditions must be true in order for the step to be performed. If you want to evaluate conditions separately, you’ll probably need to create extra pipelines. Which then introduces a problem of how do you limit the triggering of the various pipelines? I found this a hard problem to solve, though it was something that could be worked around.

Something to keep in mind/that bit me for a while if you’re adding multiple pipelines: the --- between your pipelines is 100% necessary.

kind: pipeline
type: docker
name: backend

steps:
- name: build
  image: golang
  commands:
  - go build
  - go test

--- <-----YOU NEED THIS
kind: pipeline
type: docker
name: frontend

steps:
- name: build
  image: node
  commands:
  - npm install
  - npm test

Another slightly annoying limitation with triggers is while it support triggers based on tags, there’s no support for releases.

Environmental Variables

Drone has a lot of different ways to handle/import variables. There are several ways to import/use secrets, including into plugin settings or as environmental variables. Another trick is that you can add variables as you would normally in bash with something like var=Test and calling $var in your commands.

The frustration is that it’s not possible to share variables across pipelines (so you have to import it every time). What I ended up doing a number of times is piping the variable to an .env file and then sourcing it every stage as needed. Again, there’s a solution here, it just isn’t always obvious or easy to get to (and sometimes feels like a kludge).

Git Pushes

Oh. My. God. This was probably the most frustrating thing I dealt with when setting up Drone. There’s an official Git Push plugin available but with a combination of some long standing issues like this one and this one I just could not get the plugin to work nicely with committing back to GitHub with SSH. I even tried something like this where I would add the SSH key as an environmental variable but I kept hitting fatal: could not read Username for 'https://github.com': terminal prompts disabled. Also, I could not figure out how to push git tags with the official plugin which is something I needed.

The solution ended up being surprisingly simple. So simple I’m shocked that it’s not included in the documentation anywhere. Since Drone uses HTTPS to clone the GitHub repo, the easiest way to do a commit back is over HTTPS as well. All you need is to generate a Personal Access Token for GitHub (I have not tested/tried this with Gitea or the other options so YMMV) with the public_repo scope. Add the token as a secret in Drone, and then do something like this:

  - name: generate tag
    image: alpine/git
    environment:
      GH_API_KEY:
        from_secret: push_api_key
    commands:
      - git remote add github https://YOUR_USERNAME:$GH_API_KEY@github.com/USER/REPO.git
      - git tag -a 0.0.1 -m "Release v. 0.0.1"
      - git push -u github main 0.0.1

You can modify the above to also push git commits or really any other git based action. Again, I don’t understand why this isn’t documented given how common a problem pushing with SSH is.

Conclusion

In the end, I think a large part of the problem is that the documentation could be clarified. For example, here it says “Action support varies by event and SCM provider” but fails to explain where I might find out more about what the SCM providers might support. Another recent issue that I noticed is that Harness moved the GitHub repo for Drone to their own. I’m not sure if they also closed down the Issues section or if that was done before, but now a bunch of links on the Drone forum to various issues just 404. Kinda sucks losing that.

There also are some who have issues with the licensing of Drone. The Drone Server is open sourced under an Apache2 license while the Runners appear to be propriety/not open source. Personally, I don’t mind this part as much as it’s still self-hostable and free though I do understand the concerns. If you only want to use open source software, there’s an alternative that forked Drone and appears to be under active development. Can’t say I’m a fan of the name (Woodpecker? really?).

This turned out to be way longer than intended, and it may seem like I really don’t like Drone. In fact, I both love and hate Drone. The flexibility it offers is awesome, it has a simple and clean UI that’s relatively easy to navigate, and I personally am a fan of YAML. There’s just a couple of pain points that honestly just…shouldn’t be pain points. They seem so simple to fix. And I wish I could program in GO but I cannot help.

Using Renovate-Bot to Manage Infrastructure

Tue, 09 Nov 2021 11:01:00 +0000

Combined with using Ansible, I’ve also begun using Renovate-Bot to update my infrastructure GitHub repo. My main use case is to update the tags on my Docker containers. I’ll be honest and I’ve barely scratched the surface of what Renovate-Bot can do. But having tried various means of notifying and updating me of updated docker containers and I’ve found Renovate-Bot to be the best. Below are the solutions I’ve tried and the pros/cons of each.

Updating Docker Containers

Watchtower

There are many ways to update your docker containers. One of the more popular approaches is to tag your containers using latest and then run Watchtower. Watchtower is another docker container that can run on a predefined period and will automatically update the containers you specify. This approach has a ton of appeal. It’s the most “set and forget” approach and ensures you’re always running the latest container version. However, the main drawback (and it bit me many times) is an upgraded latest container that breaks/doesn’t work. Nothing like waking up in the morning to find several of your critical containers have stopped functioning. With Docker it’s relatively easy to roll back and get back up and running, but sometimes that can be a pain. Particularly if a database is involved and the updated version is incompatible with earlier versions.

DIUN

After Watchtower, I moved on toDIUN (Docker Image Update Notifier). DIUN can also run on a pre-defined schedule and was the solution I ran for a long time. It’s a really well-developed tool and is highly flexible and customizable. Basically it monitors the various Docker container registries (such as Docker Hub and the GitHub Container Registry) on a cron-based schedule. If it detects a new container image it can send a notification to any of a number of notification services (Telegram, Slack, Gotify, etc).

My annoyance with DIUN isn’t really DIUN’s fault. DIUN tracks updates based on whether the hash of the container has changed. If you’re running the latest I ran into some containers where DIUN would say the container was updated but in fact it wasn’t. Mostly this would happen with multi-arch containers. Eventually I got tired of getting notifications telling me there was an update when there wasn’t.

I could solve most of the issues by moving to a different tag (such as a semver appropriate one like 4.4 or something like stable) but sometimes the only tag available for an image is latest. Also, by pinning to a specific version unless I was paying close attention I would miss that a container had moved to, let’s say 4.5 instead of 4.4.

Ultimately I still run DIUN for a few images that don’t have good tags available and just run latest. It’s not ideal but it works for the few where that’s a problem.

Renovate-Bot

By far the best solution I have found is Renovate-Bot. Renovate is like Dependabot on GitHub in that it automatically can update dependencies as specified. What makes it better than Dependabot is it is way more flexible. It supports a ton of languages and backends (GitHub, Gitea, Gitlab, etc.). Really I’ve only begun to understand a tiny fraction of what it can do and I’m very much impressed. I mean, just check out all the configuration options! Best of all, it’s free and open source.

My usecase is to manage the version tags of my Docker containers. Renovate-Bot automatically creates pull requests that detail the version it would update to and in some cases a link to or a description of the changes based on the changelog. Then I just have to merge the PR, pull it down to my local repo and then push the update out to the appropriate server. It took a little work to get my configuration right to track the versions but once I got it it really just works. And best of all, I don’t have to monitor all the various containers/repos to be notified when there’s an update.

There are many ways to run Renovate-Bot. Basically you can either self-host it (yes there’s a Docker container) or run it via Renovate’s service. Personally I’m using the GitHub App option instead of GitHub Actions but I’ve been exploring self-hosting it eventually. Maybe once I deploy Drone and some more CI stuff I’ll get around to it.

Conclusion

In the end, there’s no “right” or “wrong” way to manage container updates (though I’d argue Watchtower is borderline on the “wrong” way TBH). I know some people have scripts that notify them via Discord or Telegram and they can respond and have the bot update the container. I know other people just run latest and update as they remember in one big update with a docker-compose pull && docker-compose up -d. In the end, you’ll need to figure out what’s best for you.

Ansible - Swiss Army Knife of the Homelab

Tue, 09 Nov 2021 10:13:58 +0000

For the past 6 months or so I have slowly fallen in love with a tool called Ansible. In fact, pretty much every computer in my possession is controlled and managed by Ansible. I wanted to do a writeup on my usecases/setup for Ansible, which also will lead to another post shortly on using something called Renovate-Bot to help upkeep some of my code. Together my setup has pretty much become self-maintaining. Though admittedly that hasn’t stopped me from continuing to make improvement/tweaks.

Introduction

What is Ansible? SImply put, Ansible is a tool that allows you configure and manage practically any computer device that you can SSH into. It’s very well documented and has an extensive community supporting it. Best of all, it doesn’t require installing any special software on the target machine other than Python. It may require you to install certain modules on the target machine to run certain modules but that’s all documented. I use Ansible for two related reasons. First, I use it as a combination of personal wiki/knowledgebase. It’s nice not to have to remember “how the heck did I set up Wireguard again” or “what’s that application I used to do X?”. Second, I use it as a form of backup. It’s not really itself a backup but I have the vast majority of my setup in Ansible, meaning should I need to rebuild a server I can do it quickly with pretty much two commands.

Ansible is kind of like a Swiss Army knife. It can be used to do pretty much anything you can think of. Maybe some of them aren’t the greatest ideas but the point is it’s incredibly flexible. As you can see from my Infrastructure GitHub repo, I have a number of roles that are dedicated to individual server setups (like copying over specific bash scripts or installing needed packages). I also have a number of roles that can automatically install and configure applications, like Restic/Autorestic or Syncthing just the way I need.

In addition to roles you create yourself, as I said above the Ansible community is fairly awesome. Chances are someone has already written a role to do many of the things you’re trying to accomplish. You can find may of them on Ansible Galaxy or just be searching Google/DuckDuckGo and finding some GitHub repos. There’s also others who have their entire infrastructure code available on GitHub like mine that you can poke through to see what’s possible/people are doing.

Setting up Ansible

I don’t want to go into a ton of detail on the actual setup of Ansible. There’s tons of good guides out there, like this one by Jeff Geerling and this one that I used on LearnLinuxTv. Honestly, a guide on Ansible would be….much longer than I’m prepared to write. There’s a lot to learn, and I’m by no means an expert, but once you get the basic idea it’s pretty easy to go from there.

There are a number of ways to install Ansible. It’s available in most package repositories and also as a Python package on PyPi. Depending on your distribution I’d suggest installing via pip because there’s a fair number of releases a year and the packages can be woefully behind (looking at you Ubuntu/Debian). Again, the main requirement here is Python.

Beyond that, I would suggest you start by reading/watching a few guides to understand the basics of how Ansible operates. The basic gist of it is that you create various Playbooks which can run a number of tasks or roles. Tasks are the basiic building blocks of Ansible. You use individual modules likecopy or templateto accomplish your goals. A role is basically a set of pre-packaged tasks; you can use it as a shorthand to call a series of tasks you want to use in a variety of places.

Probably the most important part of Ansible to learn is variables. Variables allow you to reuse text over and over again easily and also to be able to use a role on multiple devices and override the variable as necessary. For example, let’s say in general I want to create a user named fuzzy so I set a {{main_username}} variable as fuzzy. But on one machine I want the user created to be named elmo. All I’d need to do is set the default on that one machine to elmo and I’m all set. Figuring out and setting variables and your defaults is probably the biggest thing to focus on in my opinion.

Infrastructure As Code

Once you get your Ansible playbooks the way you like, if you’re feeling brave open source your code to give back to the community. I’m a fan of the Infrastructure as Code movement. As I mentioned above, I have my entire Ansible setup available on a public GitHub repo. I love the Ansible community and love how open it is and how much people are willing to help and provide examples. Having my repo available is my way of giving back/saying thank you. But wait you might say. Fuzzy, what about your SSH keys! Your usernames and passwords! Surely you don’t have those publicly available.

Why yes, yes I do. Welllll…..sorta. If you feel like to crack Ansible’s AES256 bit encryption, have at it. It might take you a while. To do this, you’ll need to set up Ansible Vault. Here’s a well-written guide I followed to set up my vault.

Now, not going to hide the fact that I’ve been burned before. One night, I was up late. I had made a ton of changes and wanted to commit the changes to GitHub before going to bed. I ran the commit and started to head up to bed. As I’m walking upstairs I get a ping from GitHub. “Oh hey, we just noticed your SSH keys to connect to GitHub were published on GitHub.” Then another one from Cloudflare stating they’d come across an API key. Crappppppppp. I’d forgotten to encrypt my Ansible vault prior to committing it to the repo. And being GitHub there’s really no good way to delete a commit absent nuking the entire repo. So at 1am that’s what I had to do. And since I couldn’t exactly leave my setup running with the literal keys to the kingdom (SSH keys, passwords, etc) having been exposed, I spent the next 2 and a half hours changing every password/SSH key that had leaked. It was NOT an experience that I care to repeat.

As a result I took two main actions. First, I triple checked my pre-commit Git script to make sure it worked by checking if my Ansible vault is encrypted before I can actually commit to GitHub. Second, I make a point of only decrypting my vault when I need to make changes. Admittedly when the oops happened I was making a lot more changes than I am now that required me being in the vault, but now my default is to encrypt the vault whenver I’m done for the evening. Don’t be me, learn from my mistake.

Conclusion

Apologies if this post is a little all over the place, but I really just wanted to write up on Ansible and just how much I enjoy using it. It’s really change my life when it comes to my home lab and combined with Proxmox I’m really happy with the state of things. It’s not a tool for everyone but I encourage everyone to at least check it out.

Universal, Synced Retrogaming Setup

Thu, 04 Nov 2021 04:32:58 +0000

And now for something entirely different! While I’m not the biggest gamer, I do appreciate playing games from time to time, both retro games and more modern games. I’m very very excited to be picking up a SteamDeck and am lucky enough to be getting one in December. I also have a computer hooked up to my TV and my desktop

RetroArch

The dominant program in the retrogaming space is RetroArch which utilizes the libretro API. RetroArch has tons of “cores”, or emulators, which can emulate pretty much any gaming system you can think of. Basically every retro gaming front end/program that I’m aware of/have tried relies on RetroArch as a backend to actually play the game. I’m not going to cover much of installing/configuring RetroArch itself as that’s a whole post in and of itself. I use Ansible to modify a few of the basic settings I want (like hiding certain menu options) and downloading the cores I prefer but beyond that RetroArch “just works.”"

Frontends

There’s a number of frontends you could use to select and play your games which can utilize RetroArch as a backend. Things like Lakka. Batocera, Recalbox, and RetroPie are popular options. All have limitations however. Lakka/Batocera/Recalbox use stripped down OS’s that don’t allow much flexibility in terms of installing other software (like syncthing as discussed below) which I found to be a dealbreaker. RetroPie is really meant to run on a Raspberry Pi and while there’s a script to install it on Ubuntu I’m not the biggest fan of the RetroPie UI as it’s not internally consistent and settings aren’t grouped together. The above solutions may work for you so I’d suggest checking them out but ultimately were not what I was after.

RetroArch

The easiest solution here would be to use the default/built in GUI in RetroArch. And it’s….fine. There’s a few different themes (I like the now-default “Ozone” theme) and you can do a fair amount of tweaking of the retroarch.cfg file once you find the right setting (or change it in the UI). Honestly it would work in a pinch but it just didn’t “wow” me by any means.

RetroArch uses playlists to create lists of games you can play. I found a python script that can generate the playlists for you and then I manually pulled the appropriate game art into the thumbnails directory. But I found that to be a pain to do

The biggest issue for me was that I also wanted to be able to easily open/play some of my Steam games (either directly or via SteamLink) or something like Minecraft. There’s no way to do that from the RetroArch UI (or at least not easily that I could find), which meant I’d have to pull out a keyboard to launch them. Not ideal. So I went off in search of alternatives.

Pegasus

I found a few options in this space. Launchbox is Windows only (boo) so was out. GameHub was a real contender but it didn’t seem to have the ability to go true “full screen.” I decided to keep looking.

After searching a bit more, I stumbled across Pegasus-Frontend which literally checked every box for me. It allows for launching third party apps like SteamLink or the Minecraft launcher. It has some nice themes. And it easily integrated with RetroArch/retro games.

Figuring out the configuration was a bit tricky. The files are saved to ~/.config/pegasus-frontend and specifically a settings.txt and game_dirs.txt file, as well as a metadata folder. Note that you’ll want to create individual folders inside the metadata folder with a metadata.pegasus.txt per folder. So for example, I have:

pegasus-frontend/
└── metadata/
    └── roms/
        |-─ atari2600/
			├── metadata.pegasus.txt
			└── media (optional)
		|── nes/
			├── metadata.pegasus.txt
			└── media (optional)

Steam games are automatically imported. To get retro games automatically added I used Skyscraper. Configuration is a bit beyond the scope of this post but you can take a look at my Skycraper config file and the bash script I use to scrape/import the media into Pegasus. Basically I run that bash script and everything gets generated. Note that the launch command in the config.ini file is the key as that is used to call RetroArch and launch the games. You can customize RetroArch to exit when you close the game, which then takes you right back to Pegasus.

Like I said above, Pegasus has a lot of themes. The default one is pretty nice. I really liked the GameOS theme though.

Syncthing

Now we have a nice way to view/find the games, how do we get the games themselves onto the various devices and ideally sync save files/progress so I can stop playing on one device and pick up playing on another right where I left off.

The easiest solution to this problem would be a simple network share using Samba or NFS. This way you don’t use up storage on the device and everything is easily accessible. There’s a few issues with this approach though. First and most importantly, with something like the SteamDeck where I can be on the move and off my WiFi network I would lose access to the games/save files. Second, a potential issue, particularly if you’re using a Raspberry Pi using an SD card, would be data corruption. Using a network share there’s a chance the file may get corrupted and synced back to the central storage location. This is probably pretty unlikely but it’s possible.

I came across a solution that I like which is using syncthing. Credit for this idea goes to Jay from LearnLinuxTV. Syncthing is a binary written in GO which is highly customizable and efficient in syncing data between multiple devices. And it solves the two issues above because it syncs the data directly to the other device and also allows for one way sync (so no bitrot/corruption issue).

Syncthing is supposed to “just work” and be able to detect other devices running syncthing and allow you to share folders easily. Personally I could never get the automatic detection to work (possibly because of my VLAN/firewall rules), which is fine because it’s possible to set it up with IP addresses.

To make this easier, I created a syncthing Ansible role that 1) installs/updates syncthing as a GO binary directly and 2) creates a configuration file you can customize to pre-define shares. I didn’t want to rely on a PPA/package manager because those can be notoriously out of date. For configuration, syncthing uses an XML file for configuration so it’s pretty easy to modify the file, reload syncthing and the settings will be updated. What I do is sync the games folder one way from my NAS/server to my desktop/TV box/SteamDeck and then sync save files between all the devices. Now all my save files are synced up so I can pick up right where I left off on any device. And it all happens seamlessly.

Conclusion

In the end I now have a unified way to view and play basically all my games. It took me probably longer to figure out how to do all this than I care to admit but I’m pretty happy with the result. Really excited for my SteamDeck and looking forward to playing with it once I get it (which I guess now is February….).

Backing up your data with Restic (+ Autorestic) and Minio

Thu, 02 Sep 2021 08:38:00 +0000

Note: See update here. I no longer use or recommend Minio with Restic. There’s a better option!

Having a good backup strategy has always been important to avoid data loss but is now even more important in this day and age of constant ransomware stories in the news. Despite your best efforts to avoid clicking anything, all it takes is one slipup or mistake and your data can be lost. I’ve long hunted for a good backup strategy and I think I’ve finally settled on a pretty good option utilizing Restic (plus a wrapper called Autorestic, more below) backing up data to my Synology NAS and to Backblaze’s B2 Cloud Storage.

Backup Strategies

Before getting too far into the weeds, let’s talk basics about backup strategies. For a long time, the advice has been 1 is none, two is one, and three is good. IE having one copy of your data is as good as having none, two copies means you really have just one, and three is a good start. This advice can be summarized with the 3-2-1 backup strategy, where you have three copies of your data on two different media with one being offsite. I think this generally is sound advice to follow, though personally I don’t think two different forms of media is as important for home/personal use. Let’s be honest, how many of you have tape backups of your data? I bet more than 90% of people just have the data backed up to hard drives. So

Recently, Backblaze wrote a fantastic article arguing that it may be time to rethink the 3-2-1 strategy because ransomware is growing more clever and is going after backups. So you don’t want your backups to be accessible and be offline. The article discuss a 3-2-1-1-0 and 4-3-2 strategies. Personally I follow a sort of hybrid where I have four copies of my data backed up to 3 different devices, one of which is offsite and an extra copy that’s stored offsite AND offline (so I guess 4-3-1-1?). I have the “live” copy of my data, plus a copy backed up regularly to my Synology and to B2, and then most of my most important data gets backed up to an external HDD that lives in my office desk drawer. The external drive only comes home once a month for an update and then goes straight back to the office.

Restic

At first my solution was to use Duplicati. It offered a GUI plus configurable backups that were incremental and on a scheduled basis. But I had so many issues with it where my backup database would get corrupted and I’d have to wipe everything and restart. Or the backups would fail silently and I’d get no warning about it. There’s some ways to work around it but overall it’s not a tool I would recommend.

For a while I switched to doing rsync backups of my data to various places but there were numerous issues with this strategy. If my data got encrypted then it would just get copied over to the backup and overwrite the “good” data. I had no version control. And it was horribly inefficient. I tried to solve some of the issues by making tarballs of my data but then I ran into storage issues because there was no deduplication of the data so I was exponentially increasing the amount of space required to backup my data as I went. Eventually I stumbled onto Restic and it basically solved all my issues. There’s a few other options in this space, BorgBackup and Kopia as other popular solutions.

What I like about Restic is that it supports a wide variety of backup methods, such as local disks, SFTP, and S3 Object Storage. It also encrypts all your backups and supports incremental backups, though it does not support compression (yet, maybe someday). You can set up retention policies so you can manage how long your data is backed up, and importantly forget and prune unneeded data. During a forget/prune, Restic will go through your data and repack it so that it only keeps the data it needs and forgets the snapshots it no longer needs. Overall I’ve been very impressed with the performance. I have about 500GB of data backed up via Restic (to both the Synology and B2) and prunes take at most a half hour on my largest datasets. Most of the time it’s minutes.

Autorestic

This next part is entirely optional, but I think it makes the Restic usage experience much simpler. Autorestic is a wrapper for Restic that makes the configuration and basic commands very easy to execute. There’s TONS of options out there for Restic wrappers, such as Crestic and restique. Pick one that works for you and has the features you want. But the rest of this guide is for using Autorestic.

Follow the installation guide for Autorestic here. Then you’ll need to create a configuration guide. The documentation is pretty good here, and feel free to poke around my Github Infra repo and take a look at my autorestic config files, like this one. The basic idea is there’s a backend (S3, local, SFTP etc) and a location which is the location of the data you’re backing up to a backend. You can have multiple backends and backup a location to multiple backends (though you can only have a single file location/path per location). You set the backup password and that’s pretty much it. You can further configure to run some hook scripts (such as calling Healthchecks.io), setting up a retention policy, and excluding files/directories

As I mentioned above, I backup my data to my Synology (using Minio) and Backblaze’s B2 service. I found these options easier to configure than either setting up a storage mount (which would make it easier for ransomware to encrypt) to use local storage or even SFTP which required sharing SSH keys which can be a PITA.

Minio

Minio is an S3 compatible object storage app that can be run in Docker, and thus on the Synology NAS. This is now much easier on DSM7, the latest version of the DSM software as it ships with a fairly recent version of Docker. I would highly encourage you to install docker-compose on the Synology see the docs here as Synology’s Docker GUI is….well…bad. It’s functional but to change things (like the command variable needed for Minio) you have to delete and recreate the container from the GUI. Docker-Compose just makes it all simpler and easier. Here’s my docker-compose for Minio:

version: "2"
services:
  minio:
    image: minio/minio
    container_name: minio
    volumes:
      - /volume1/Minio:/data
    ports:
      - 9000:9000
      - 9001:9001
    environment:
      - MINIO_ROOT_USER=YOUR_USER
      - MINIO_ROOT_PASSWORD=YOUR_PASSWORD
    command:
      ['server', '/data', '--address', '0.0.0.0:9000', '--console-address', '0.0.0.0:9001']
    restart: unless-stopped

Once you set up Minio, you can access it at YOUR_SYNOLOGY_IP:9000 in your web browser. Login with your root user/password (this is what you use for your AWS_ACCESS_KEY_ID and SECRET_ACCESS_KEY for Autorestic). Alternatively, Alex/IronicBadger has a post here for setting up a reverse proxy (using Traefik) to access Minio. I prefer the IP address but to each their own.

And that’s it! You now have an S3 compatible object storage location on your Synology NAS. Makes backup pretty easy if you ask me!

Conclusion

I’ve been running this setup for about 4 months and so far it’s been rock solid. I’ve had to restore data a few times and the data has been there just as I needed it. And because the backups are incremental my B2 storage bill is around $2/month for about 500gb of storage. $2 well spent for some peace of mind over ransomware.

Using Wireguard to Tunnel All Traffic through a VPS to Home

Thu, 26 Aug 2021 20:38:00 +0000

In my last post, I discussed how I was moving off of Cloudflare and also moving to Caddy. After about a month of completing that switchover, I’m sticking to it. Still have a few issues with the way Caddy does things but overall it works.

In this post I want to discuss my Caddy setup, particular how I am not directly exposing my homelab/server to the internet but instead am routing all the traffic through a VPS. Conceptually it’s pretty simple, but it took me a while to actually implement. There’s many solutions out there for implementing a similar setup and there may be a simpler way to do what I’m doing but my way works so I’m not messing with it.

Overview

So why route everything through the VPS? A few reasons. First, I don’t have to expose my home server to the internet. That means that there are no ports open on my home firewall, particularly not ports 80/443. Second, I don’t have to reveal my home IP address to the whole world being a DNS record. And finally, I don’t have to worry about a dynamic DNS updater failing and losing access to my services should my IP address change. In essence, this provides me with a lot of the same benefits of Cloudflare but without being on Cloudflare. The downside is that it’s more complicated and has some more running parts, any of which could break and would bring down remote access to my apps, but I think the benefits are worth it.

So how do I do it? Here’s an image that explains it:

Basically traffic comes into the VPS, gets routed by a Caddy server running on the VPS down a Wireguard tunnel to a server running on my LAN in a DMZ. The DMZ server also runs a Caddy server and routes the traffic to the appropriate app server. For the record, yes, I know I could have used something like Nebula or Tailscale or Zerotier and built a mesh network where everything was interconnected. However, two things kept me from going down that path. First, I didn’t want to to have to set up/manage multiple connections to the VPS. More things that could possibly break. Second, I wanted to route everything through a single, well-hardened and secured server before crossing into my home network. And third, many of the mesh VPN options out there are either not open source or require you to use a proprietary server as the “main” hub. Nebula is an exception on both counts and I highly recommend reading this post if you’re interested in setting up Nebula, but it still was overkill for my needs as I just wanted a single tunnel/connection to worry about.

Setup

Now that we’ve talked about the why, let’s talk about the how. For this you’ll need a VPS, a reverse proxy (the examples below will be in Caddy but NGINX would work just fine too as would Traefik I suspect), and Wireguard. Not sure I’ve really ever mentioned Wireguard on this blog before but it’s amazing. If you’re still using OpenVPN just….stop. Wireguard is a hell of a lot more efficient and far easier to set up. There are tons of tools for configuring it and loads of GUIs you can chose. For this though I’m configuring it all manually.

Wireguard Setup

Installing Wireguard is fairly straightforward, just follow the instructions on the Wireguard page or check out one of the many, many blog posts/guides out there like this one. Once it’s installed, we need to create the tunnel. For that, you’ll need two sets of public/private keys. Generating them is pretty simple, the hardest part is keeping track of which key goes where. Hopefully the below example configuration files help make that clear. You’ll need to save the files in /etc/wireguard. Personally I saved mine as wg0.conf.

VPS Wireguard Conf:

[Interface]
Address = 10.10.10.1/24
ListenPort = 51820
PrivateKey = PRIVATE_VPS_KEY

[Peer]
PublicKey = PUBLIC_DMZ_KEY
AllowedIPs = 10.10.10.10/32

DMZ Wireguard Conf:

[Interface]
Address = 10.10.10.10/32
PrivateKey = PRIVATE_DMZ_KEY
DNS = 9.9.9.9, 149.112.112.112

[Peer]
PublicKey = PUBLIC_VPS_KEY
Endpoint = VPS_IP_ADDRESS:51820
AllowedIPs = 10.10.10.1/32
PersistentKeepalive = 25

Probably don’t need the DNS entries but figured it couldn’t hurt. You definitely want the PersistentKeepAlive to ensure that the connection remains open and doesn’t close/nothing gets blocked. You can change the IP address (in my case 10.10.10.1/24) to any private IP address range you want, but I liked the IP of the DMZ being 10.10.10.10. Easy to remember/type. I also limited the IP addresses to just those on the tunnel, otherwise you run into issues where DNS won’t resolve, no internet, etc. Plus, it’s the only traffic you need to route.

Once you created your config files on both servers, run sudo systemctl enable wg-quick@wg0.service and sudo systemctl start wg-quick@wg0.service. You can check the status with sudo systemctl status wg-quick@wg0.service and also trying to ping each end of the tunnel (so from the VPS ping 10.10.10.10 and on the DMZ ping 10.10.10.1). You should see successful pings. If not, check your firewall rules.

To ensure that the Wireguard tunnel stays up, I modified a script I found that pings the IP address of the VPS on Wireguard (in my case, 10.10.10.1). If that fails 3 times, it reboots the Wireguard systemd service. That obviously won’t help if the VPS is offline but it would help with any temporary odd blips. I added a cronjob to run the script every 5 minutes.

#!/bin/bash
# Modified from https://mullvad.net/en/help/running-wireguard-router/
# and https://wiki.r-selfhosted.com/guides/virtual-private-networks/wireguard/
# ping Wireguard gateway to test for connection
# if no contact, restart!

PING=/bin/ping
## DEBIAN
SERVICE=/usr/sbin/service

tries=0
while [[ $tries -lt 3 ]]
do
    if $PING -c 1 10.10.10.1
    then
            echo "wg works"
            exit 0
    fi
    echo "wg fail"
    tries=$((tries+1))
done
echo "wg failed 3 times - restarting tunnel"
## DEBIAN
$SERVICE wg-quick@wg0 restart

Caddy/Reverse Proxy Setup

Getting the Wireguard tunnel working was probably 90% of the battle for me, so I’m not going to heavily detail the reverse proxy part. Plus it will depend on what reverse proxy you’re using. If you have questions feel free to contact me and I’m happy to try to help/discuss!

Here’s my example Caddyfile on my Infra GitHub repo. The basic gist would be the same in NGINX, basically all you do is tell the reverse proxy to send the traffic to the DMZ server’s Wireguard IP address. On the DMZ Server, here’s my Caddyfile. The DMZ Caddy Server listens on port 80 at the URL you want, and then redirects the traffic to the appropriate server on the LAN.

As you can see, I terminate SSL on the VPS and route everything internally using HTTP. Some I know prefer to terminate SSL on the homeserver/DMZ, which is valid but I just found it simpler/more straightforward to do it on the VPS.

Conclusion

Now there are some downsides to this approach. The biggest one I ran into was that Fail2Ban no longer worked when running on the individual app servers on my LAN. The reason was that Fail2Ban would attempt to ban the correct external IP address but iptables only cared about the Wireguard IP address. IE Fail2Ban would add 100.40.39.38 to the banned iptables list, but iptables would only see traffic coming from 10.10.10.10 or 192.168.50.10 so the ban wouldn’t be effective. To get Fail2Ban working I had to implement rsyslog to send the various logs up to the VPS and then run Fail2Ban on the VPS. It works but it still feels like a hack and it would have been much simpler if I could have just kept running Fail2Ban on the individual servers. I’m intrigued by something like CrowdSec but haven’t had a chance to implement it yet.

The other thing to keep in mind is you’ll need to configure some of your apps to handle a trusted proxy, otherwise the IP address it will see is that of the DMZ server or the Wireguard tunnel.

This approach really works best if you aren’t funnelling tons of traffic through the VPS. Not because the VPS can’t handle it from a performance perspective but because most VPS providers cap your data. Linode, for example, allows 1TB a month on the $5 tier. For me that’s plenty but if you’re routing lots of say Jellyfin/Plex traffic through it you may want to consider a different approach (or directly sending heavy bandwidth apps directly to your LAN).

Overall, despite some struggles to get this set up, it’s been rock solid for me and I really like the way it’s running.

Switching DNS Providers and Reverse Proxies - What could go wrong?

Wed, 04 Aug 2021 08:32:00 +0000

So this post is going to be a bit different than usual. No tutorial or guide, more of a discussion of some changes I’ve made on the backend to host this blog and some of my other self-hosted services. Recently I’ve started using Ansible to deploy all my servers (you can see my setup here). I didn’t convert my Linode VPS instance to Ansible because I didn’t want to screw up how I had things. But as I detailed below, I wanted to switch my DNS servers for my domains off Cloudflare. I wanted to test the setup before making any permanent changes so I spun up a second Linode VPS. And since I was doing that I figured why not just Ansibile-ize it?

So that’s how I got here, but why the change? Let me explain.

Cloudflare/DNS Servers

I’ve been a big fan of Cloudflare for a long time. They provide a tremendous product with great value for home users, including a CDN, DDoS protection, free SSL certs, and DNS servers just to name a few things. Pretty much since my early days of self-hosting things, they have been my go-to. I appreciated that I could point my domains to my home IP address and everything would be proxied/anonymized. It may not really have gotten me any protection but it made me feel better. I also appreciated that I had a level of DDoS protection from bots.

But as they say, if the product is free, you are the product. Now that arguably may not be true in Cloudflare’s case (I expect their goal is to convince people to tell their bosses to switch to Cloudflare at work and make money from the enterprise sector), but I’ve steadily grown more uncomfortable with the sheer amount of dominance that Cloudflare has over things. Also, with Cloudflare handling the SSL certs and providing the CDN, it means that Cloudflare decrypts all traffic to its servers before reencrypting it when it leaves. From a privacy standpoint, I slowly became less and less comfortable with this concept as well. Not that there have been any reports of Cloudflare doing anything nefarious with data but it just didn’t sit right with me anymore in this era of “big data.”

So what is the alternative? Well, there are a few. My basic requirements were 1) free, 2) support DNS-01 ACME challenge for Let’s Encrypt/ZeroSSL certs, and 3) support an API of some kind to allow dynamic DNS updates in case my home IP address changes (ideally with OPNsense as that’s what I use). There were a few options out there that fit the bill. Ultimately I ended up switching to Linode’s DNS Manager. Linode uses Cloudflare for DDoS mitigation, load balancing and distributed name servers. But as best I can tell, NOT it’s CDN so my traffic isn’t proxied through Cloudflare’s servers.

Adding a domain to Linode is super easy and I found the documentation here perfectly clear. The major downside I’ve found with Linode is that it’s slower than Cloudflare to propagate DNS changes. With Cloudflare, I would create a new CNAME entry for example and it would be available in less than 5 minutes. Linode seems to only update their records every 15 minutes or so, which in the grand scheme of things is fine. I mean, how often do you really switch your DNS entries? Linode also doesn’t support DNSSEC yet but for my purposes that’s fine.

So far I’ve switched this domain and another off Cloudflare to Linode and have had zero issues. So I don’t see any reason to switch back.

NGINX/Reverse Proxy

So that was the DNS change. While I was doing that I figured why not check out different reverse proxies (have I mentioned I’m a bit of a technomasochist?). The main contenders I considered were Traefik and Caddy. I know many people over on the Selfhosted subreddit love Traefik and sing its praises. It’s particularly great if you run lots of Docker containers as it handles reverse proxying to those really easily just by adding some environmental variables to the container in question. I’d tried Traefik before and just…couldn’t wrap my head around it. Plus I found the configuration by labels just confusing as heck (and potentially making for a long docker-compose file!). I also found the documentation sorely lacking and unfortunately when I was trying it the devs had just upgraded from v1 to v2, making a number of breaking changes and the guides out there on the internet hadn’t been updated. Overall, just very frustrating for me. That left Caddy.

Caddy also had a major update from v1 to v2 last year and from what I can gather it was not a smooth update for all and there are still some things that aren’t quite there yet (I’ll explain more below). And the documentation still is not very clear on all things. But overall, I have found it to be remarkably simple to set up. The feature I’m loving the most so far is that a lot of the defaults are things I already want, like some basic HTTP headers. I don’t have to declare them every single time.

For example, here is my NGINX config for Healthchecks:

server {
    listen         80;
    server_name    hc.fuzzymistborn.com;
    return         301 https://$host$request_uri;
}

server {

        listen 443 ssl;
        server_name hc.fuzzymistborn.com;

        ssl_certificate      /etc/nginx/ssl/cert.pem;
        ssl_certificate_key  /etc/nginx/ssl/key.pem;
        ssl_client_certificate /etc/nginx/ssl/cloudflare.crt;
        ssl_verify_client on;
        ssl_prefer_server_ciphers on;

    location / {

        proxy_pass http://healthchecks:8000/;
        proxy_set_header Host $host;
        proxy_redirect http:// https://;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $remote_addr;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

    }

}

Yes, there’s probably something wrong in there but you’ll see my point when you look at my Caddyfile:

hc.fuzzymistborn.com

reverse_proxy localhost:8000

Ok, not quite the full file (again, I’ll explain later) but you see what I mean but how much simpler it is. Caddy 1) by defaults upgrades everything from HTTP to HTTPS, 2) can handle the SSL cert generation if needed, and 3) takes care of most of the headers for you. In some cases I still needed to add X-Real-IP and X-Forwarded-Proto but that can easily be handled. Once you wrap your head around the principles of Caddy it’s easy to see the allure and simplicity of it (and the much much shorter config file!).

It’s not all sunshine and daisies though. Here are some of my issues with Caddy. First, if I was to define each subdomain like in my example above, it would fetch SSL certs for each subdomain. Not very efficient, and while I don’t have that many subdomains, it’s enough that I don’t want to deal with it. To get around that, I wanted to use a wildcard cert. To do that, you need to utilize the handlers and host matchers, so my actual config looks more like this:

*.fuzzymistborn.com {

        @hc host hc.fuzzymistborn.com
        handle @hc {
                reverse_proxy localhost:8000
        }

}

Which then brings me to my second issue. For some reason I’m not very clear on, Caddy in v2 decided to ditch using LEGO for it’s DNS authentication and switched to libdns which is missing a LOT of providers that LEGO supports, including Linode in my case. The previous plugin is still available and can be installed but I found the documentation confusing on how exactly to implement it. Instead, I just decided to use LEGO (which btw is probably my favorite name for something as it’s for generating Let’s Encrypt (LE) certs and it’s written in GO, so LEGO is amazing) and pull the certs manually. I then configured LEGO via a cronjob to renew the certs. So I “lost” one of the main benefits of Caddy over NGINX.

A third issue is that Caddy is compartmentalized, meaning to get certain features you need to download/compile it into the binary. Things like DNS resolvers aren’t built into the default but must be added. It’s all relatively simple and I understand the reasoning behind it (makes it simpler to develop) but from an enduser it’s confusing and at times frustrating. For example, the DNS providers have to be individually added instead of in a single library/plugin.

Overall I’m not entirely sold on the benefits of Caddy over NGINX. I like having a single, clean, simple config file and now that I’ve figured some things out I appreciate the defaults/basic structure. But it was not simple and there are some definite quirks. Honestly I’m not sure if I’ll stick with it or switch back to NGINX as there is more documentation/support out there for NGINX than there is for Caddy. If you’re curious, you can see my Caddyfile over on my Ansible infra Github repo here.

What’s next for me is to set up some kind of VPN tunnel between my VPS and homelab so I don’t need to expose my homelab to the outside world, probably using something like Nebula, though I have been playing around with a Wireguard tunnel too. Once I get that going, I’ll switch my last personal domain off Cloudflare.

Using WakeOnLan with Opnsense and HomeAssistant

Fri, 28 May 2021 08:38:00 +0000

Note: Significant hat tip here to /u/abstractbarista for this guide. Most of the steps below come from their guide. The main differences being 1) little bit improved layout/format (subjective) and 2) I figured out how to do a call to just wake a single device.

Introduction

WakeOnLAN is one of those things that you love to hate. When it works, it’s magical. When it doesn’t it’s beyond frustrating. My biggest annoyance is with my network setup, I have multiple VLANs/subnets, and WOL doesn’t really like to be routed across subnets for reasons I only barely understand so I’m not going to try to explain. To get around this, my solution for a long time was to have a Raspberry Pi Zero on the subnet I wanted with the wakeonlan package. I then set up SSH keys and added it to HomeAssistant, along with a script I could call via a shell_command. It worked great but having to poke firewall rules, having SSH keys on HomeAssistant, etc always made me a little uncomfortable.

My bigger problem was that I needed a Pi Zero on the same subnet which usually wasn’t a problem. However I’ve been playing around with a homelab (my old retired server) but I didn’t want it running 24/7 wasting power when I only use it sporadically. Instead of having to shut it down/turn it back on (requiring me to trudge down to my basement) I figured I’d try using WOL and just suspending the server when I’m not using it. Well I don’t have a Wifi network for my Homelab VLAN so I can’t use a Pi Zero, and I really didn’t want to use a full Pi for something as simple as WOL.

Enter Opnsense. I’ve been running Opnsense as my firewall for about 2 months now and I have to say I love it. Coming from an Edgerouter the difference is just night and day. The main thing I’m really loving is all the plugins and tools that are available. You can do so much on your router if you run Opnsense/PFsense (depending on which is your cup of tea, not getting into the debate between the two here) it really is worth it. I discovered Opnsense has a WOL plugin and found it worked great. Worst case I could just log into Opnsense, go to the WOL service and wake up my homelab server. But that’s just too much effort when you can get it into HomeAssistant!

Opnsense Setup

Opnsense has a REST API that you can call prettily easily, particularly when combined with HomeAssistant’s REST API command integration. The documentation for the WOL API on Opnsense is….lacking. However, the Reddit post from /u/abstractbarista linked above got me 90% of the way there. Their solution was simply to wake all devices. I wanted to target mine since I have a few that I like to sleep/wake up so I kept digging a bit. Here’s the basic steps:

Install the os-wol plugin in Opnsense by going to System->Firmware->Plugins.
Go to Services->Wake on Lan and click the plus icon in the bottom right corner next to “Wake All”, and add your device/MAC address along with the interface. I think this step is optional given the later steps but it can’t hurt to add.
Next step is creating a REST API token. You could use an existing Opnsense user but I’d suggest creating a separate one just for HASS. Go to System->Access->Users->Add. Pick a username (doesn’t matter) and enter a stupidly long/secure password (you’ll probably never need it). Leave everything else as default. Click “Save and go back”.
By default the user can’t do anything. You’ll need to give it permission to access to the API. Edit the user and scroll down to “Effective Privileges”. Click the edit button and search for “Services: Wake On Lan”. Select it and then save. Then scroll down and create an API key by clicking the plus button. You’ll be prompted to download a txt file that will have the user/key you’ll need later. Save this somewhere for now. Click Save at the bottom.
Depending on how you have your firewall rules set up, you probably need to create one to allow HomeAssistant to connect to the REST API on Opnsense. I’m assuming you know how to create firewall rules for your network, but the specific rule you need is to allow traffic from your HomeAssitant instance to “This Firewall” on either HTTP or HTTPS (depending on whether you have HTTPS enabled on Opnsense). Congrats, you’re done with the setup on Opnsense. The rest is easy.

HomeAssistant Setup

As mentioned above, HomeAssistant has a REST Command integration.

rest_command:
  wake_device:
      url: 'http://OPNSENSE_URL/api/wol/wol/set'
      method: POST
      payload: '{"wake":{"interface": "INTERFACE","mac": "DEVICE_MAC"}}'
      content_type:  'application/json'
      username: !secret opnsense_user
      password: !secret opnsense_key

There are a few things to note here. First, the Opnsense URL should be either HTTP or HTTPS depending on your setup. If you use HTTPS you may need to add verify_ssl: false. The /api/wol/wol/set is the part that matters/shouldn’t change. Second, to find the value for the interface, you’ll need to go to “Interfaces” in Opnsense and figure out the designation for the interface where the device you’re trying to wake up is. Just click on the interface and look at the URL. You should see something like /interfaces.php?if=opt5. opt5 is what you want. The device MAC is self explanatory. For username and password, put the user/key from the txt file you downloaded during step 4 above. Restart HomeAssistant and you now have a new service call, rest_command.wake_device.

From here, you can use the command however you want. Personally I combine it along with a template switch and binary ping sensor so I can put my desktop to sleep/wake it up with a switch. Totally up to you where you want to go from here.

Hope someone finds this useful, and again major thanks to /u/abstractbarista for the inspiration and most of the heavy lifting!

Home Automations for Kids

Wed, 28 Apr 2021 04:32:58 +0000

No, this is not and article on how to get your kids to want to create your home automations for you. Sorry. But I’ve been meaning for a long time to document some of the home automations things I do to make life easier with a baby/toddler. Hope you find these helpful and perhaps give you some ideas of your own!

As inspiration, a popular baby item is the Hatch Baby Rest, which is a sound machine, night light, and time-to-rise/ok-to-wake clock all built into one. It’s pretty reasonably priced at $60 and is “smart” in that you can control it from your smartphone. However, I have two issues with the device. 1) it uses Bluetooth, which can be a notoriously fickle beast, and 2) because it’s Bluetooth, it means no Home Assistant control! There’s an $80 version that’s WiFi enabled here but again it’s not in HomeAssistant. So where’s the fun in that when you can DIY something yourself that’s infinitely more customizable?

Sound Machine

Introduction

My very first automation set up was a smart sound machine. For a while we were using a dumb sound machine which worked fine. The first one ran on batteries and we churned through them. We upgraded to a plugged in one but then it was well…annoying to have to turn on the sound machine like a caveman. To make a smart sound machine, I ultimately settled upon a Raspberry Pi running Volumio. This is a pretty nifty piece of software that is a great option if you want high quality sound. I wasn’t particularly interested in that aspect of it, but it checks the boxes of 1) Runs on a Pi and 2) has a HomeAssistant component that allows us to do what we need.

Parts Needed:

Raspberry Pi (a 3B or 3B+ will do. So would a 4 but that is overkill), along with a power supply
Speaker with a 3.5mm headphone jack (and a cable to connect the Pi to the speaker)
Optional (but strongly recommended): Door/contact sensor. Useful for being the trigger to turn the sound machine on/off, but can use voice controls if needed. Zwave/Zigbee/433mhz all would work here depending on your setup.
Optional: Raspberry Pi UPS. I picked this one but there are a few drawbacks. 1) it’s got some stupidly bright LEDs that I’ve had to cover with electrical tape and 2) it’s design is such that if the battery is completely drained and power is restored, the Pi won’t boot back up without manual intervention. I can live with that since if the power is out for that long and the Pi dies, I’ve got bigger issues. I also struggled to find a case that would work, ultimately deciding to leave the Pi naked.

Volumio Setup

The build here is really pretty simple. Take a Pi, get an SD card, flash it with your flashing program of choice (I like BalenaEtcher personally), insert the SD card and start up the Pi. The Volumio Getting Started Guide is pretty well done and should get you set up so the Pi is on your network (WiFi works fine for our purposes, you’ll just need to follow the steps to get the Pi on your network). There’ll be a setup guide, you should be able to just stick to the defaults.

If you want to use a network drive you can add it during the setup. But I’d suggest the following instead in case your internet/power goes out. Enable SSH following these instructions. What white noise you want to use/works best for your kid is up to you. Mine likes ocean noises. Save the file to /data/INTERNAL using your SFTP program of choice (I like FileZilla).

When you navigate to the IP address of the Pi an you’ll see a screen like this (versions may have different versions):

Unfortunately it’s been a while since I set these up so I can’t recall all the settings I needed to change, but the file we loaded to the Pi should appear in “Music Library”. You can tweak some of the other settings to your liking. The one that you need to change is in “General Playback Options.” Scroll down till you get to “Playback Mode” and make sure it’s on “Continuous” and not “Single.” Otherwise the track will just play once and not repeat (which is something you’ll want for white noise). I also set the startup volume to 80% but you’ll want to tweak based on your setup/what works for you.

Next let’s set up a playlist. Click on “Music Library” and “Internal.” You should see the file you loaded up. If you click the 3 dot icon on the far right side, you should see an option to add to playlist. Create a new playlist (call it what you want, I called mine “Whitenoise”).

I suggest playing the playlist to test that everything is working on the Volumio side. Including making sure the track repeats.

HASS Setup

Assuming everything is working on the Volumio side, let’s move to HASS. Open your HASS instance, go to Integrations and click the add button and search for Volumio. Input the IP address of your Volumio instance and everything should get set up for you. This will create a media_player entity for you. Personally I found a switch easier (will allow easier automations and voice controls), so I set up a template switch like this:

  - platform: template
    switches:
      kid_sound_machine:
        friendly_name: "Kid Sound Machine"
        value_template: "{{ is_state('media_player.kid_sound_machine', 'playing') }}"
        turn_on:
          service: media_player.select_source
          data:
            entity_id: media_player.kid_sound_machine
            source: Whitenoise
        turn_off:
          service: media_player.media_pause
          data:
            entity_id: media_player.kid_sound_machine
        icon_template: mdi:cast-connected

The above switch will be on if Volumio is playing and off otherwise. Toggling the switch will call the media_player.select_source service call. You’ll need to alter the entity_id to match your device and the source to be the name of your playlist. Alter the entity_id for the turn_off call as well and you should be good and ready to automate!

How exactly you want to automate is up to you so I’m not going to share any automation code. But here’s what I have setup. For starters, I have a “bedtime” script that gets triggered by a Zigbee Aqara button (my toddler loves to push it) that does a random TTS saying “Goodnight” and dims the lights while we read books. The script pauses until the door to the kid’s bedroom is closed, and then it shuts off the lights and turns on the sound machine. If we open the door in the middle of the night, the light turns on very dim so we can see but not blind anybody (and again shuts off if the door is closed). It doesn’t shut the sound machine off. In the morning (i.e. between 5:30 and 8:00am), if the door is opened the sound machine shuts off and the light slowly ramps up to full brightness.

I have a slightly different setup for naptime. During the daytime, if the door is closed and the light in the kiddo’s room is still on, the sound machine doesn’t turn on, though the automation loops every couple of seconds to check if the light is off. Once the light is off, the sound machine turns on. I did put a check for naptimes to not turn the sound machine off/turn on the lights during the day if the sound machine changed within the last 15 minutes, using this template: {{ ((as_timestamp(now()) - as_timestamp(states.switch.kid_sound_machine.last_changed)) / 60 | float ) > 15 }}. That way we can go if needed right after putting kiddo down. Outside of that, if the door is opened and the sound machine is on, the sound machine shuts off and the same slow light ramp up as the morning runs.

Honestly the setup has been fairly bulletproof (even when grandparents visit/watch the kiddo while mom and dad go out for a date) and has made things just a little easier for bedtime and in the morning.

Misc. Thoughts

Note in some circumstances I ran into issues where when I connected the Pi and the speaker to the same power plug I was getting ground feedback which was REALLY painful to hear. Ultimately just doing some finagling and I resolved this, so if you run into the issue try different cords/power supplies/adapters/plugs etc. Originally I was planning on using Bluetooth for the connection from the Pi to the speaker but…see my thoughts on Bluetooth and it being temperamental.

Ok to Wake Clock

This one is fairly new as my toddler started getting out of bed in the middle of the night and we wanted to try to find a way to get him to stay in bed. And also for some naps when he doesn’t sleep we wanted a way to signal to him when it is ok to get out of bed. So I designed a series of automations that 1) will turn the light red when he gets out of bed in the middle of the night (doubles as a night light) and 2) will turn the light red an hour before a time we set and then green when the time hits. Then for naptime there’s a timer that when there’s less than 10 minutes left turns the light red, and then when the timer hits zero goes green.

I can’t say it’s completely successful as my toddler has still gotten out of bed on multiple ocassions and pounded on his door until we go in and tuck him back into bed. But there have been a few times where he’s gotten out of bed, the light has gone red and he’s gotten back in so I’m going to call it a win. Hey, you take your victories with toddlers where you can!

Parts Needed:

Bed sensor (previously covered here, based on guide here)
Some kind of smart LED bulb
Optional: Door sensor (but you already have one for the sound machine right?)

HASS Entity Setup

Initially I used some input_selects to set the wakeup times, but I recently switched to using input_datetime instead and it’s much nicer. I created 2: one for “normal” wakeup times and the other for custom wakeup times in case I need to change something on the fly and don’t want to change the “normal” time.

input_datetime:
  toddler_wake:
    name: "Time"
    has_date: false
    has_time: true

  toddler_custom_wake:
    name: "Time"
    has_date: false
    has_time: true

I also created an input_boolean to toggle the custom time on/off.

To control the color of the light, I created a fairly complicated template sensor:

  - platform: template
    sensors:
      toddler_wake_trigger:
        friendly_name: 'Trigger'
        value_template: >
          {% if (state_attr('input_datetime.toddler_wake', 'timestamp') - 3600) | timestamp_custom('%H:%M', false) <= states('sensor.time') < state_attr('input_datetime.toddler_wake', 'timestamp') | timestamp_custom('%H:%M', false) %}
            Red
          {% elif (state_attr('input_datetime.toddler_wake', 'timestamp')) | timestamp_custom('%H:%M', false) <= states('sensor.time') < (state_attr('input_datetime.toddler_wake', 'timestamp') + 600) | timestamp_custom('%H:%M', false) %}
            Green
          {% else %}
            off
          {%- endif %}
      toddler_wake_custom_trigger:
        friendly_name: 'Trigger'
        value_template: >
          {% if ((state_attr('input_datetime.toddler_custom_wake', 'timestamp') - 3600) | timestamp_custom('%H:%M', false) <= states('sensor.time') < state_attr('input_datetime.toddler_custom_wake', 'timestamp') | timestamp_custom('%H:%M', false)) and states('input_boolean.caw_wakeup_custom') == 'on'  %}
            Red
          {% elif (state_attr('input_datetime.toddler_custom_wake', 'timestamp') | timestamp_custom('%H:%M', false) <= states('sensor.time') < (state_attr('input_datetime.toddler_custom_wake', 'timestamp') + 600) | timestamp_custom('%H:%M', false) ) %}
            Green
          {% else %}
            off
          {%- endif %}

To explain, if the time is between 1 hour of the time set by the input_datetime and the current time, the sensor state is Red. I use the Time&Date integration but you could also substitute in now if you wanted. If the current time is equal to or greater than the time set (up to 10 minutes) then the state is Green. Otherwise it’s off. I use this sensor state to drive a series of automations to turn the light to Red/Green/Off depending on if the kiddo is out of bed when he shouldn’t be. I’ll have a copy of my NodeRed flows at the end of the post so you can see what I’m doing.

The other thing I have is a timer that controls naptime. When the door is closed during the day, an automation calls the timer.start service with a duration set by an input_select, though I subtract 10 minutes off. When the timer ends, an event is generated by HASS that you can then trigger off of, which I use to set the light to red for 10 minutes and then turn green.

Lovelace Setup

For my dashboard, I relied on the awesome Time Picker Card, with a final product that looks like this:

Simple but functional.

Conclusion

If you have any other great parenting automations I’d love to hear! This post turned out to be longer than I anticipated and I may have left some things out. If you are at all interested in more information on my setup, please feel free to hit me up on Discord, Reddit, or open a Github issue on my blog’s repo. But overall these automations have assisted me in parenting by handling menial tasks (like controlling lights and sound machines) so I can focus on the kiddo.

Automation

Here’s my NodeRed flow for the Ok To Wake part:

[{"id":"c70c60fa.07d298","type":"time-range-switch","z":"b06c555f.69395","name":"","lat":"","lon":"","startTime":"19:00","endTime":"05:45","startOffset":0,"endOffset":0,"x":570,"y":4400,"wires":[["bf7349b0.84f1a8"],[]]},{"id":"50d4e1aa.35ce68","type":"api-call-service","z":"b06c555f.69395","name":"Nightlight Red 50%","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"light","service":"turn_on","entityId":"light.toddler_lamp","data":"{\"color_name\":\"red\",\"brightness_pct\":\"50\",\"transition\":\"1\"}","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":950,"y":4420,"wires":[[]]},{"id":"580415a.1f37fec","type":"api-call-service","z":"b06c555f.69395","name":"Nightlight Off","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"light","service":"turn_off","entityId":"light.toddler_lamp","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":930,"y":4460,"wires":[[]]},{"id":"bf7349b0.84f1a8","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"on","vt":"str"},{"t":"eq","v":"off","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":675,"y":4400,"wires":[["50d4e1aa.35ce68"],["580415a.1f37fec"]],"l":false},{"id":"8c020bf5.2132b8","type":"server-state-changed","z":"b06c555f.69395","name":"Toddler Door","server":"63517380.eb951c","version":1,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"binary_sensor.door_toddler_bedroom","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":true,"for":"","forType":"num","forUnits":"minutes","ignorePrevStateNull":true,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"x":250,"y":4400,"wires":[["12a622ef.49a37d"]]},{"id":"12a622ef.49a37d","type":"api-current-state","z":"b06c555f.69395","name":"Sound","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"switch.toddler_sound_machine","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":410,"y":4400,"wires":[["c70c60fa.07d298"],[]]},{"id":"e04cfc1b.c8318","type":"server-state-changed","z":"b06c555f.69395","name":"Toddler out of bed","server":"63517380.eb951c","version":1,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"binary_sensor.toddler_bed_sensor","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":true,"for":"","forType":"num","forUnits":"minutes","ignorePrevStateNull":true,"ignorePrevStateUnknown":true,"ignorePrevStateUnavailable":true,"ignoreCurrentStateUnknown":true,"ignoreCurrentStateUnavailable":true,"x":270,"y":4500,"wires":[["ad0c136a.3b262"]]},{"id":"1a97fdb0.6f7e1a","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"on","vt":"str"},{"t":"eq","v":"off","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":515,"y":4500,"wires":[["ee3c11ad.ab3f4"],["7a9130e7.19b018"]],"l":false},{"id":"3c1cf6ba.8db91a","type":"server-events","z":"b06c555f.69395","name":"Naptime Over","server":"63517380.eb951c","event_type":"timer.finished","exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"waitForRunning":true,"x":250,"y":4700,"wires":[["4ca59a1d.91eb04"]]},{"id":"4ca59a1d.91eb04","type":"switch","z":"b06c555f.69395","name":"","property":"payload.entity_id","propertyType":"msg","rules":[{"t":"eq","v":"timer.toddler_nap","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":390,"y":4700,"wires":[["6ff4e0f4.16a788"]]},{"id":"ad0c136a.3b262","type":"api-current-state","z":"b06c555f.69395","name":"Sound","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"switch.toddler_sound_machine","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":430,"y":4500,"wires":[["1a97fdb0.6f7e1a"],[]]},{"id":"6a0240f6.f4479","type":"api-call-service","z":"b06c555f.69395","name":"Nightlight Red 20%","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"light","service":"turn_on","entityId":"light.toddler_lamp","data":"{\"color_name\":\"red\",\"brightness_pct\":\"20\",\"transition\":\"1\"}","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":950,"y":4560,"wires":[[]]},{"id":"813a29ca.756498","type":"server-state-changed","z":"b06c555f.69395","name":"Toddler Normal Wake","server":"63517380.eb951c","version":1,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.toddler_wake_trigger","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":true,"for":0,"forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"x":280,"y":4560,"wires":[["552e2175.3a42e"]]},{"id":"92f176f0.e80ea","type":"api-current-state","z":"b06c555f.69395","name":"Custom Off","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.toddler_wakeup_custom","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":630,"y":4560,"wires":[["f66c8284.fe6128"],[]]},{"id":"f66c8284.fe6128","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"Red","vt":"str"},{"t":"eq","v":"Green","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":745,"y":4560,"wires":[["6a0240f6.f4479"],["d8bef19b.c0597"]],"l":false},{"id":"d8bef19b.c0597","type":"api-call-service","z":"b06c555f.69395","name":"Nightlight Green 30%","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"light","service":"turn_on","entityId":"light.toddler_lamp","data":"{\"color_name\":\"green\",\"brightness_pct\":\"30\",\"transition\":\"1\"}","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":960,"y":4620,"wires":[[]]},{"id":"1dd70335.de05dd","type":"server-state-changed","z":"b06c555f.69395","name":"Toddler Custom Wake","server":"63517380.eb951c","version":1,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.toddler_wake_custom_trigger","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":true,"for":0,"forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"x":280,"y":4620,"wires":[["7faf5a5f.a5726c"]]},{"id":"7aa5433f.5e89ac","type":"api-current-state","z":"b06c555f.69395","name":"Custom On","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.toddler_wakeup_custom","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":630,"y":4620,"wires":[["c1edd371.4f068"],[]]},{"id":"c1edd371.4f068","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"Red","vt":"str"},{"t":"eq","v":"Green","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":745,"y":4620,"wires":[["6a0240f6.f4479"],["d8bef19b.c0597"]],"l":false},{"id":"2468b11d.f25156","type":"delay","z":"b06c555f.69395","name":"","pauseType":"delay","timeout":"10","timeoutUnits":"minutes","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":670,"y":4700,"wires":[["d8bef19b.c0597"]]},{"id":"7a9130e7.19b018","type":"api-render-template","z":"b06c555f.69395","name":"Green?","server":"63517380.eb951c","template":"{{ state_attr('light.toddler_lamp', 'rgb_color') != (0,255,0) }}","resultsLocation":"result","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":620,"y":4520,"wires":[["deb18a84.7afe4"]]},{"id":"deb18a84.7afe4","type":"switch","z":"b06c555f.69395","name":"","property":"result","propertyType":"msg","rules":[{"t":"eq","v":"True","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":715,"y":4520,"wires":[["6a0240f6.f4479"]],"l":false},{"id":"ee3c11ad.ab3f4","type":"api-render-template","z":"b06c555f.69395","name":"Checks","server":"63517380.eb951c","template":"{{ states('binary_sensor.door_toddler_bedroom') == 'off' and ( states('sensor.toddler_wake_trigger') == 'off' and states('sensor.toddler_wake_custom_trigger') == 'off' ) }}","resultsLocation":"result","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":620,"y":4480,"wires":[["6bd1c50f.179e34"]]},{"id":"6bd1c50f.179e34","type":"switch","z":"b06c555f.69395","name":"","property":"result","propertyType":"msg","rules":[{"t":"eq","v":"True","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":715,"y":4480,"wires":[["580415a.1f37fec"]],"l":false},{"id":"552e2175.3a42e","type":"api-current-state","z":"b06c555f.69395","name":"Sound","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"switch.toddler_sound_machine","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":450,"y":4560,"wires":[["92f176f0.e80ea"],[]]},{"id":"7faf5a5f.a5726c","type":"api-current-state","z":"b06c555f.69395","name":"Sound","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"switch.toddler_sound_machine","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":450,"y":4620,"wires":[["7aa5433f.5e89ac"],[]]},{"id":"6ff4e0f4.16a788","type":"api-current-state","z":"b06c555f.69395","name":"Sound","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"switch.toddler_sound_machine","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":510,"y":4700,"wires":[["2468b11d.f25156","6a0240f6.f4479"],[]]},{"id":"63517380.eb951c","type":"server","name":"Home Assistant","legacy":false,"addon":false,"rejectUnauthorizedCerts":false,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

New Blog, Who Dis?

Sat, 10 Apr 2021 01:58:58 +0000

So for the past year, my blog has been run on the Ghost blogging platform. And I really liked it. It was simple, it worked, and I liked the look. Why change you might ask? Well…first I have a problem where I constantly find myself wanting to tinker, even if there’s a perfectly working thing. It can always be better right? Also, Ghost just released a major update (v4) which indicates, at least to me, they’re taking the platform in a different direction from just being a blog platform. Specifically they’re catering more to people who want to create newsletters/have subscriptions/generate revenue which….isn’t why I blog. I blog for fun and to document things that I do so others can learn/benefit from my suffering…err….experience. And when I upgraded there was this big huge “subscribe” button in the middle of the header (which even more annoyingly 404’d when clicked):

I don’t want to sound like I’m hating on Ghost, the platform is great and works very well. The direction they’re heading just isn’t for me. Sooo….new shiney time!

Enter Hugo. It’s a static-site generator, which basically means it creates the webpages based on flat files. There’s no database or running programs or anything. You can find the entire source code for all the posts in this Github repo. Which I find pretty cool. Particularly because a few of my posts seem to be popular but in some cases may be out of date or could be improved. So if you spot something that’s incorrect or needs to be updated, please open up an issue on the blog Github repo, or better yet, make a PR to correct it! Also feel free to suggest topics for me to cover by opening an issue, and I’ll post some of my ideas on there as well and I welcome feedback.

Bear with me over the next few weeks as I fix up some things and may make some other changes or other improvements. I hope this will make things better in the long run but for now there may be some growing pain.

Also, thanks to this blog post for detailing how to migrate from Ghost to Hugo: https://blog.jimdhughes.com/migrating-my-blog-from-ghost-to-hugo/

Controlling Eufy Security Devices with ioBroker

Mon, 28 Dec 2020 22:55:00 +0000

Note: I don’t or recommend Eufy’s hardware anymore. I think Amcrest or Reolink are more compelling options that actually have working RTSP (even over WiFi) for just a bit more money. Also, if you still want to use Eufy, you don’t need to use ioBroker anymore. See this GitHub repo for more info.

As I think should be obvious, I’m very big on HomeAssistant. It’s a great piece of software and does many many many things well. However, I bought a Eufy Doorbell camera over a year and a half ago and genuinely love it. It has great set of features (local person detection, fast notifications, quick loading of streams in app, 2k video quality) but it lacked a HomeAssistant integration. I created a repo to document the API but Eufy changed things around and it grew well beyond my skill to understand. There’s a working integration for HomeAssistant but it’s limited to getting still images of last motion activity and creating a 3 minute RTMP stream from the camera (limitation of the API, not the integration).

But then there’s this “adapter” for ioBroker. I will admit to not knowing much about ioBroker other than what you can glean from their website and some Googling. It’s a predominantly German IoT platform that in many ways is similar to HomeAssistant. It is able to connect to a lot of popular smart home devices. And again, most importantly for my purposes, thanks to the work of @bropat it’s able to control my Eufy Doorbell camera (and other Eufy indoor cameras).

In this post I will walk you through how to install ioBroker via Docker, set up the EufySecurity adapter, and then connect to it via HomeAssistant.

ioBroker Setup

This guide is for setting up ioBroker in Docker. If you’re running HassOS, ~~you’ll need to look elsewhere because there’s no way to run ioBroker currently inside of HassOS. Maybe someday. But for now, you’ll need to use another machine to run the container.~~ Check out @MaxW’s HassOS add-on: https://github.com/MaxWinterstein/homeassistant-addons/. Note that it’s EXPERIMENTAL. You have been warned.

For the Docker image, we’ll be using https://github.com/buanet/docker-iobroker. This seems to be the “official” image and it works well enough for me. Here is my docker-compose:

  iobroker:
    image: buanet/iobroker:latest
    container_name: iobroker
    ports:
      - "8081:8081" # Main
      - "8082:8082" # Web Server
      - "8087:8087" # Rest API
    volumes:
      - /home/fuzzy/docker/iobroker:/opt/iobroker
    environment:
      - LANGUAGE="en_US:en"
    restart: always

Adjust the volume mount path (in my case /home/fuzzy/docker/iobroker) to whatever suits your purposes.

There are a lot more things you can set via Docker environmental variables (see the Github page for more details) but this is the bare minimum that will get a container spun up and working.

Run docker-compose up -d iobroker and the image should download and start. Go to http://yourip:8081 and you should see this screen:

Click that you agree to the terms and click “Agree.” Then enter your information or click “close” (this won’t really matter for purpose of EufySecurity but if you want to mess around with ioBroker feel free to fill it out “correctly”). You’ll then be taken to an “instance” page. Ignore this for now and click on “Adapters” in the left-hand column.

Then click on “filter” and search for “Simple Restful API”. Click the blue 3 dots and then the plus sign in the bottom left corner. Let it install/add the instance. You can ignore the settings page that appears after installation.

Go back to the adapters page. I also installed the “WEB server” adapter which will let you pull the last image from the eufy-security adapter into HASS, so this step is optional if you want to continue using the current HASS integration.

But this time click on the Github icon in the top left corner. Click “Custom” and then enter the URL of the ioBroker EufySecurity Github: https://github.com/bropat/ioBroker.eufy-security.

You should now have the EufySecurity adapter installed. Go back again to the adapters page and you should see a EufySecurity box. Click on the blue circle with 3 dots and then click the plus button in the bottom left corner (exactly the same steps as we did to add the Restful API adapter). It will create an instance for you. Here you will want to fill in your information to set up the adapter. You can change the polling interval (I have mine set to 5 minutes instead of the default 10) and the max camera livestream duration (default is 30 but I believe the API should support up to 3 minutes or 180 seconds). Click “Save and Close” and you should see the circle next to “eufy-security.0” turn green eventually (may start red then go yellow and eventually green).

HASS Integration

Now the fun part! Integrating with HASS. For this you’ll need to dig around a bit in the “Objects” page of ioBroker. Look for a “folder” called eufy-security.0. Expand that folder and you should see a list of all your cameras available via the adapter. They’re listed by ID/Serial Number but also have the name you set up in the app. You’ll need the serial number to get things into HASS.

Let’s start with the guard mode. Hat tip to @MaxW on the HASS forum for this part. For each camera you want to track the guard mode status of, create a RESTful sensor in the sensor: portion of your HASS configuration.yaml file. For example:

sensor:
  - platform: rest
    name: Eufy Doorbell Guard Mode
    resource: "http://IOBROKER_SERVER_IP:8087/getPlainValue/eufy-security.0.CAMERA_SERIAL_NUMBER.station.guard_mode"
    scan_interval: 10
    value_template: >-
      {% set v = value | replace('"', "")|int  %}
      {{ {0: "Away",
          1: "Home",
          2: "Schedule",
          3: "Custom1",
          4: "Custom2",
          5: "Custom3",
          47: "Geofence",
          63: "Disarmed"}[v] | default("Unknown state:" + (value|urlencode)) }}

Change the Server IP address and camera serial number to match your server/camera

This will get you the current status (at least, it polls every 10 seconds so close enough) of the guard mode of each camera. To change the status, you’ll need to create a rest command. Here’s an example config:

rest_command:
  eufy_doorbell_disarmed:
    url: "http://IOBROKER_SERVER_IP:8087/set/eufy-security.0.CAMERA_SERIAL_NUMBER.station.guard_mode?value=GUARD_MODE_VALUE&ack=false"

Change the Server IP, Camera Serial Number, and Guard Mode as appropriate. You’ll also want to make as many commands as you want to be able to change to.

Now there’s two options for how to change the settings. First, you could create a template switch that would go back and forth between two modes. That’s good enough for my use case as all I really want is “motion detection on” and “motion detection off.” So here’s my template switch:

  - platform: template
    switches:
	  eufy_doorbell:
        friendly_name: "Front Door - Guard Mode"
        icon_template: mdi:doorbell-video
        value_template: "{{ is_state('sensor.eufy_doorbell_guard_mode', 'Home') }}"
        turn_on:
          service: rest_command.eufy_doorbell_home
        turn_off:
          service: rest_command.eufy_doorbell_disarmed

The other option I could see is creating an input_select and then an automation to fire depending on which input you selected. Whatever floats your boat. I may go down the input_select route eventually but for now the switch does the job well enough.

The final step is if you want to pull the last motion image into HASS. This is where the WEB Server comes into play. Here’s my camera config:

camera:
  - platform: generic
    name: Eufy Doorbell
    still_image_url: http://IOBROKER_SERVER_IP:8082/eufy-security.0/CAMERA_SERIAL_NUMBER.jpg

I’m working on a way to get the RTMP stream into HASS. I’ve figured out how to get the URL (via the RestfulAPI) but I haven’t quite cracked the camera part yet. The URL I get back has quotes and that doesn’t seem to work in HASS. Edit: Got it working! Hat tip to @dezmui for cracking the quotes part and the camera part. Here’s how to do it:

Add the following as another rest sensor for any stream you want.

  - platform: rest
    name: Eufy Doorbell Stream
    resource: "http://IOBROKER_SERVER_IP:8087/getPlainValue/eufy-security.0.CAMERA_SERIAL_NUMBER.cameras.CAMERA_SERIAL_NUMBER.livestream"
    scan_interval: 10
    value_template: >-
      {% if (value) is not none and (value) != 'error: datapoint &quot;Livestream URL&quot; not found' %}
        {{ value.replace('"', "") }}
      {% else %}
        No Stream
      {% endif %}

And then this as a camera component (and you also need to enable stream: in your configuration.yaml!). Note that this is very similar to the above camera setup, we just added the stream_source as an option.

camera:
  - platform: generic
    name: Eufy Doorbell
    still_image_url: http://IOBROKER_IP:8082/eufy-security.0/CAMERA_SERIAL_NUMBER.jpg
    stream_source: "{{ states('sensor.eufy_doorbell_stream') }}"

Now to start and stop the stream (subject to the limitation that the stream will time out based on what you have your setting in ioBroker set to), use the rest_commands:

rest_command:
  eufy_start_stream:
    url: "http://IOBROKER:8087/toggle/eufy-security.0.CAMERA_ID.cameras.CAMERA_ID.start_stream"

  eufy_stop_stream:
    url: "http://IOBROKER:8087/toggle/eufy-security.0.CAMERA_ID.cameras.CAMERA_ID.stop_stream"

Restart HASS and then add this to your Lovelace:

  - camera_image: camera.eufy_doorbell
    entities:
      - entity: camera.eufy_doorbell
    title: Camera
    type: picture-glance
    double_tap_action:
      action: call-service
      service: rest_command.eufy_start_stream
    hold_action:
      action: call-service
      service: rest_command.eufy_stop_stream

Should work!

Conclusion

So there you have it. Now you can control much of the functionality of EufySecurity cameras via HomeAssistant. Not all functionality is supported but @bropat seems to be doing a great job fixing things and keeping on top of it.

What is not working for me (yet) is doorbell motion events/button pushes (even via MQTT). Those are less important to me as my main thing was controlling when the doorbell camera would detect motion or not, but I’m sure @bropat will keep working on it.

Post-Dark Sky Weather (Part 2) - Acurite and RTL433

Sat, 14 Nov 2020 02:35:09 +0000

Continuing on my last post regarding the end of DarkSky, I promised I’d follow up on the other part of how I am getting weather data into HomeAssistant. I’ve always been interested in having my own personal weather station to get hyper-accurate local weather conditions, namely temperature and wind speed. I was always off put by the need for one of those monitor stations because it restricted where and how I could see the data.

Pretty, but not great for HomeAssistant uses!

Instead I wanted the ability to get the data into HomeAssistant. Thankfully there are a lot of great options out there. Two options I considered were Ecowitt GW1000 as well as an Ambient Weather station. Both seem like good options. What put me off was the wifi connection which in some cases required a cloud connection, which I try to avoid when there’s a good non-cloud alternative.

Thankfully I found an inexpensive, non-cloud version with the Acurite 5-in-1 Weather Station. The beauty of this device is that it communicates via 433mhz, which is a fairly common standard. You could use a weather station like I have pictured above, or you could grab a Software Defined Radio (SDR) (more on this below) and thanks to some open source software, intercept and decode the signals and get the information easily into HomeAssistant.

Won’t win any awards for design

RTL_433

As mentioned above, you’ll need a SDR in order to pick up on the signals. These are pretty easy to find on Amazon. I used this one, but really any should do the trick. I tested things out on a Raspberry Pi for development purposes, but really any Linux box will work. Connect the small set of antenna to the Y splitter and connect the USB cable to the computer to get started. Then you’ll need the software to decode the signals.

Thankfully, someone has already done all the work in decoding the signals for the Acurite. RTL_433 is an absolute gem of open source software that is incredibly powerful. It may seem super complicated to a novice, but you don’t need much to get everything set up. Here’s the commands you need to enter into terminal to get RTL_433 compiled and set up:

# packages needed to compile rtl_433
apt-get update
apt-get install -y rtl-sdr librtlsdr-dev librtlsdr0 git automake libtool cmake

# Pull RTL_433 source code from GIT, compile it and install it
git clone https://github.com/merbanan/rtl_433.git
cd rtl_433/
mkdir build
cd build
cmake ../
make
make install

To test things out, you can just run the rtl_433 command and see what happens. More than likely you’ll just see a bunch of noise and intercept pretty much every signal on the 433mhz spectrum, which is….a lot. But use this to make sure you can see signals from your Acurite weather station. You can further narrow down the types of signals picked up by using the presets incorporated into the RTL_433 library. Specifically, this is the command I use to only pick up on signals from Acurite 5-in-1 weather stations: rtl_433 -C si -M newmodel -R 40.

Once you’re sure you can see the signals, it’s time to work on getting the information into HomeAssistant. The easiest way to do it is via MQTT. I’m going to assume you have an MQTT broker/server already set up with HomeAssistant (if you don’t, just Google it and there’s plenty of guides out there). I tried the example MQTT to HASS script in the RTL_433 repo, but just couldn’t get it to work. Instead, I used this script which was recommended by a friend. All you need to do is save the config.py.example and rtl2mqtt.py files to a folder, modify the config.py file to include the MQTT topic and other information, and potentially modify line 14 of the rtl2mqtt.py file. Mine is this: rtl_433_cmd = "/usr/local/bin/rtl_433 -C si -M newmodel -R 40 -R 41 -F json" (the -R 41 will make sense when you review the “Bonus Topic” below).

Again, the exact method doesn’t really matter, so long as you can find a way to get the data from RTL_433 to HomeAssistant.

HomeAssistant Setup

So now that we (hopefully) have the information running from RTL_433 to MQTT, the final step is to incorporate the sensors into HomeAssistant. You’ll need to create a number of MQTT sensors. The exact format will depend on your MQTT topic that RTL_433 is sending the data to.

Here’s the way I have the 5 sensors imported into HomeAssistant (goes in your configuration.yaml file). Note that I convert things to Imperial from Metric, hence the conversion calculations:

sensor:
  - platform: mqtt
    state_topic: "rtl_433/Acurite-5n1/1606/temperature_C"
    name: "Acurite Outside Temp"
    unit_of_measurement: "°F"
    device_class: temperature
    unique_id: Acu_Outside_Temp
    value_template: >
      {{ (value | float * 1.8 + 32) | round(2) }}
  - platform: mqtt
    state_topic: "rtl_433/Acurite-5n1/1606/humidity"
    name: "Acurite Outside Humidity"
    unit_of_measurement: "%"
    device_class: humidity
    unique_id: Acu_Outside_Humidity
    value_template: >
      {{ value }}
  - platform: mqtt
    state_topic: "rtl_433/Acurite-5n1/1606/wind_avg_km_h"
    name: "Acurite Outside Windspeed"
    unit_of_measurement: "mph"
    icon: mdi:weather-windy
    unique_id: Acu_Outside_Windspeed
    value_template: >
      {{ (value | float * 0.621371)| round(2) }}
  - platform: mqtt
    state_topic: "rtl_433/Acurite-5n1/1606/wind_dir_deg"
    name: "Acurite Outside Wind Direction"
    unit_of_measurement: "°"
    icon: mdi:compass
    unique_id: Acu_Outside_Winddirection
    value_template: >
        {{ value | round(0) }}
  - platform: mqtt
    state_topic: "rtl_433/Acurite-5n1/1606/rain_mm"
    name: "Acurite Outside Rain Total"
    unit_of_measurement: "in"
    icon: mdi:weather-rainy
    unique_id: Acu_Outside_Rain_Total
    value_template: >
      {{ (value | float * 0.03937) | round(2)  }}

This should be fine for everything but the rain sensor, which collects the rain on a cumulative basis. So if you want tracking that actually makes sense (i.e. a daily or monthly tracking), I would suggest using the Utility Meter integration. It’s really easy to set up, just add the following your configuration.yaml:

utility_meter:
  acurite_outside_rain_today:
    name: Rain Today
    source: sensor.acurite_outside_rain_total
    cycle: daily
  acurite_outside_rain_month:
    name: Rain This Month
    source: sensor.acurite_outside_rain_total
    cycle: monthly

You’ll now have a daily and monthly rain sensor that will automatically reset daily or monthly (on the 1st of the month). Incorporating these into HomeAssistant then is totally up to you!

My only other suggestion might be to create a statistics sensor to try to somewhat level out the wind speed readings. Otherwise it’s liable to look like this:

With the statistics sensor, you get a graph more like this:

Here’s my sensor entry (again goes under sensor: in configuration.yaml):

  - platform: statistics
    entity_id: sensor.acurite_outside_windspeed
    name: Recent Windspeed
    sampling_size: 30
    max_age:
      minutes: 30
    precision: 1

Bringing it all together

So if you followed my last blog post, you know I concluded I need to use a hodgepodge of weather services in order to match the information that was available via DarkSky. If you want a weather entity that combines all of this information together, take a look at https://github.com/xannor/hass_weather_template. It’s a great little custom component that lets me merge all my weather entities together so I get the best of all the worlds:

weather:
  - platform: weather_template
    weather:
      combined_weather:
        temperature_template: "{{ states('sensor.acurite_outside_temp') }}"
        temperature_unit_template: "{{ state_attr('sensor.acurite_outside_temp', 'unit_of_measurement') }}"
        condition_template: >
          {% if states('weather.nws') != 'unavailable' %}
            {{ states('weather.nws') }}
          {% elif states('weather.nws') == 'unavailable' %}
            {{ states('weather.accuweather') }}
          {% endif %}
        humidity_template: "{{ states('sensor.acurite_outside_humidity') }}"
        wind_template: "{{ states('sensor.acurite_outside_windspeed') }}"
        wind_bearing_template: "{{ states('sensor.acurite_outside_wind_direction') }}"
        attribution_template: "Weather provided by Acurite 5-1 Outdoor Sensor, the National Weather Service, and Accuweather"
        visibility_template: >
          {% if states('weather.nws') != 'unavailable' %}
            {{ state_attr('weather.nws', 'visibility') }}
          {% else %}
            Unavailable
          {% endif %}
        forecast_template: "{{ state_attr('weather.accuweather', 'forecast')[0] | to_json }}"

Bonus Topic

Well since I’m already discussing RTL_433 Acurite stuff, I might as well toss in a recommendation for these Refrigerator/Freezer temperature sensors, also made by Acurite and that also work on 433mhz. I have 2 sets as I have a fridge/freezer in my garage and have had a few instances where the door in the garage fridge has been left accidentally open and we’ve had to toss everything inside.

Weather in a post DarkSky world

Mon, 09 Nov 2020 14:53:38 +0000

Back on April 1st, DarkSky announced that it had been purchased by Apple and would be shutting down both it’s Android app and API. The API will continue to function “through the end of 2021,” but there are no new signups permitted.

This is extremely disappointing as DarkSky had a fantastic API and very generous limits for free use (1k a day, so you could easily pull updates every 10-15 minutes without issue). It also was super accurate. Once the API goes away, it will be sorely missed as there’s no solution like it out there that I have found that 1) has a free API and 2) is mostly accurate, and 3) has as much information available. While I know I still have over a year left before the API goes away, since it IS going away I figured I needed to start exploring my alternative options.

The main things I was looking for were: 1) weather forecast for the next couple of days, 2) cloud cover (use this for a few spots were I don’t have light sensors but want some kind of threshold for my motion senor lights), 3) precipitation (ie is it raining/how much), and 4) high/low temperature and a bonus for wind chill/apparent temp.

Options out there

Pulling up the list of HomeAssistant weather integrations, there seem like a decent number of options. A few require hardware though (like Ecobee, Neatmo, and Ambient) so not for everyone. A few are also international only/don’t focus on the USA which further limits the options. The ones I ended up trying were: 1) AccuWeather, 2) OpenWeatherMap, and 3) the National Weather Service. None are as good as DarkSky and in the end I’ve had to cobble together multiple services to accomplish what DarkSky easily accomplished.

Accuweather

AccuWeather is a great service (it’s the weather app I use on my phone). However, the API is extremely limited (only 50 calls a day, or twice an hour) and extremely limited information/sensors. In terms of the data provided and what I needed, it hits most of the boxes. However, the lack of long-term weather forecast (limited to only current weather unless you want to further limit your updates to once an hour) and the limited updates kinda kills it for me. If it allowed just 50 more calls a day (100) so I could update every 15 minutes I might go for it. The lowest paid tier is $25/month which is way too expensive for what I need.

Sensors available

Opinion: Accuweather is a viable, though extremely limited, option due to the API limitations.

OpenWeatherMap

This one seemed like it would be the solution. Open data with a decent API (60 calls a minute!). It also has a few different ways of configuring it:

Per the docs, the mode options are “hourly for a three-hour forecast, daily for daily forecast, or freedaily for a five-day forecast with the free tier.” I chose “free daily” as I really was looking for a long-term forecast, which also generated a lot of sensors similar to what DarkSky used to provide. I like that things seem to update frequently but I’ve also run into issues where sensors appear to get stuck (for instance, it says the cloud coverage here has been at 0% for 19 hours, which is possible but I doubt it considering I see a few clouds out right now).

Many more sensor this time

Opinion: Like the extra API calls though question the data accuracy.

National Weather Service

This one is great if all you are looking for is a forecast. Ultimately a lot of the data used to power the other weather services out there comes from NOAA/NWS so it’s fun to get the data straight from the source. However, as the integration stands right now, there are no sensors associated with it, though you can create your own with some templates as the raw data is all there.

This is all you get….

Here’s an example of a high and low temperature sensor you can create:

  - platform: template
    sensors:
      nws_high_temp:
        friendly_name: "High Temperature"
        device_class: temperature
        unit_of_measurement: "°F"
        unique_id: high_temp
        value_template: >
          {% if states('weather.nws') != 'unavailable' and state_attr('weather.nws', 'forecast')[0].daytime == true %}
            {{ state_attr('weather.nws', 'forecast')[0].temperature }}
          {% elif states('weather.nws') != 'unavailable' %}
            {{ state_attr('weather.nws', 'forecast')[1].temperature }}
          {% else %}
            Unavailable
          {% endif %}
      nws_low_temp:
        friendly_name: "Low Temperature"
        device_class: temperature
        unit_of_measurement: "°F"
        unique_id: low_temp
        value_template: >
          {% if states('weather.nws') != 'unavailable' and state_attr('weather.nws', 'forecast')[0].daytime == false %}
            {{ state_attr('weather.nws', 'forecast')[0].temperature }}
          {% elif states('weather.nws') != 'unavailable' %}
            {{ state_attr('weather.nws', 'forecast')[1].temperature }}
          {% else %}
            Unavailable
          {% endif %}

Again, it’s possible to make sensors but it requires some extra work. I also have run into some issues where on start up I get errors and every now and then the API goes down or there are issues.

Opinion: Best for forecast, but terrible for sensors and has some reliability issues.

Conclusion

So which one did I pick? As I mentioned above, I chose to create a frankenstein solution built on a few of these options, combined with my own weather station in my back yard. I chose the NWS integration for the forecast, and I created a few sensors based on the data as well (high/low temperature and chance of rain). I then used OpenWeatherMap for some of the sensors I was looking for (cloud cover, precipitation type), though now that I’m looking at Accuweather again I might switch to that one instead. Just as a quick aside, how none of the integrations offered a high/low temperature sensor is mind boggling to me.

I’ll have another post up soon regarding my weather station and how I brought all of these pieces together. Unfortunately, my quest for a DarkSky replacement led me to realize there was no adequate replacement and I had to roll my own. I truly wish I could say that there was a 1-1 replacement, or even something that was 75% as good. But in reality, DarkSky was a rare gem and is going to be sorely missed when it goes away.

#RIPDarkSky

Improving Amazon Package Tracking with Tasker

Sun, 06 Sep 2020 01:10:09 +0000

First off, I am a HUGE fan of the Mail and Packages custom component. Major hat tip to @moralmonkey and @firstof9 for all their work getting this running and improving it constantly. Especially @firstof9. Whenever I find a bug/something isn’t working right he’s incredibly responsive and works hard to fix it. I find it incredibly handy to know when packages and mail might arrive that day so it’s nice to have HomeAssistant track it for me.

I won’t go into details of how to set up the component in general since I think the process is fairly well explained in the wiki. The short version of how it works is that it periodically scans your email inbox for email messages from the various carriers, looking for specific messages about packages being out for delivery. You can then fire off various automations (such as a notification) when the sensor values change.

Amazon Packages

For a long time, I was content with just having USPS, UPS and FedEx package tracking. Since all three rely on email notifications for package updates, it worked fine for me. However, recently (last month or so) Amazon has expanded their own delivery services to my area. Amazon unfortunately handles their delivery messages differently. They only send out an app notification for “out for delivery” and “almost there” updates, followed by a “your package was delivered” email along with a photo of the package on your doorstep/wheverever the Amazon driver dropped it. The latest beta version of the component is able to process the delivered message but can only do a “best guess” for when a package is out for delivery based on the initial “your package will arrive on x date” in the shipping email.

I came up with a workaround using one of my favorite Android apps, Tasker, and a plugin app for Tasker, AutoNotification. Both apps are paid (but IMO well worth the price) and this won’t work on iOS (sorry). My setup uses NodeRed and the Node-Red custom component. There’s probably a way to achieve this with HASS automations but I’m not sure how. It also requires either a NabuCasa subscription or having your HASS instance be accessible from outside your home network since it relies on webhooks. Also, the Amazon Shopping app has to be installed for the notifications to be received in the first place.

Tasker Setup

Let’s start with getting Tasker set up. First step is to set up a webhook. You can do this with NodeRed via a node or you can set up an automation in HASS. As detailed below, I did mine in NodeRed. Once you get your unique webhook ID, save it to a text file somewhere as you’ll need it shortly.

Here’s a link to the XML files you should be able to just import into Tasker. Save them to your phone, open up Tasker, and long press on “Profiles.” That should open up a menu where you can click “Import Profile.” Browse to the right profile and import it. Check out this guide if you need help.

Note that for my HTTP calls in the task, my URL is %HA_ADDR/api/webhook/%NR_WEBHOOK. Tasker allows you to use variables. Global variables (which can be used in all tasks) are in ALL CAPS. I use variables because it makes it very simple to change in case I ever need to change my HASS URL and/or the webhook ID. Otherwise I’d have to change a bunch of tasks which would be a PITA. If you don’t want to do this you can simply modify the URL to match your HASS instance URL and the webhook ID. So something like https://hass.yourdomain.com/api/webook/YOUR_WEBOOK.

Provided you’ve given both Tasker and AutoNotification the correct permissions (I found AutoNotification to be a bit tricky to get set correctly, I suggest running some tests to make sure it’s working), that should be all you need to do. You should not need to change anything else.

NodeRed Setup

Like I indicated above, you’ll need to have the Node-Red custom component installed. The reason is that the below code uses the integration to create and update custom sensors. There’s probably a way to create sensors in HASS via automations but I’m not familiar with how to do that.

Note that I went in and modified the entity_ids of the sensors once they were created to sensor.mail_amazon_packages_nearby and sensor.mail_amazon_packages_delivering. You don’t have to do this step but you’d need to modify the templates in the NR setup to match.

Here’s what your result should look like (the code to import is below):

Just to explain. When a webhook is received, if it’s an “out for delivery” or “nearby” notification it is sent via the first switch node to the correct path to then add 1 to the existing sensor value and then update the sensor. Once a package is delivered, it subtracts 1 from the sensor. I had to add a check on the subtraction because sometimes I wouldn’t receive an “out for delivery” or “nearby” notification for every package and I didn’t want negative values. Finally, there’s a reset node to trigger at midnight so everything starts off at 0.

From here you can modify your notifications to be whatever you want. I have a trigger node on mine to delay things by a minute as sometimes I have multiple packages delivered at the same time and didn’t want 5 notifications when it’s really just one delivery.

[{"id":"33f49d42.466b7a","type":"ha-entity","z":"1fe83a7f.e0f0b6","name":"Amazon Nearby","server":"63517380.eb951c","version":1,"debugenabled":false,"outputs":1,"entityType":"sensor","config":[{"property":"name","value":"Mail Amazon Packages Nearby"},{"property":"device_class","value":""},{"property":"icon","value":""},{"property":"unit_of_measurement","value":"package(s)"}],"state":"payload","stateType":"msg","attributes":[],"resend":true,"outputLocation":"","outputLocationType":"none","inputOverride":"allow","x":880,"y":4260,"wires":[[]]},{"id":"b43c5eca.57be58","type":"api-render-template","z":"1fe83a7f.e0f0b6","name":"Add 1","server":"63517380.eb951c","template":"{{ states('sensor.mail_amazon_packages_nearby') | int + 1 }}","resultsLocation":"payload","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":530,"y":4260,"wires":[["33f49d42.466b7a"]]},{"id":"e3aaae46.d1ab3","type":"change","z":"1fe83a7f.e0f0b6","name":"Reset","rules":[{"t":"set","p":"payload","pt":"msg","to":"0","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":4300,"wires":[["4b7d0e4.83c8bf","33f49d42.466b7a"]]},{"id":"a76f2d8.99415d","type":"switch","z":"1fe83a7f.e0f0b6","name":"Amazon","property":"payload.amazon","propertyType":"msg","rules":[{"t":"eq","v":"nearby","vt":"str"},{"t":"eq","v":"out for delivery","vt":"str"},{"t":"eq","v":"delivered","vt":"str"}],"checkall":"true","repair":false,"outputs":3,"x":370,"y":4300,"wires":[["b43c5eca.57be58"],["a6adb473.794bb"],["54b61769.9755f","2bd4738b.515f9c"]]},{"id":"4b7d0e4.83c8bf","type":"ha-entity","z":"1fe83a7f.e0f0b6","name":"Amazon Delivery","server":"63517380.eb951c","version":1,"debugenabled":false,"outputs":1,"entityType":"sensor","config":[{"property":"name","value":"Mail Amazon Packages Delivery"},{"property":"device_class","value":""},{"property":"icon","value":""},{"property":"unit_of_measurement","value":"package(s)"}],"state":"payload","stateType":"msg","attributes":[],"resend":true,"outputLocation":"","outputLocationType":"none","inputOverride":"allow","x":890,"y":4340,"wires":[[]]},{"id":"a6adb473.794bb","type":"api-render-template","z":"1fe83a7f.e0f0b6","name":"Add 1","server":"63517380.eb951c","template":"{{ states('sensor.mail_amazon_packages_delivering') | int + 1 }}","resultsLocation":"payload","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":530,"y":4340,"wires":[["4b7d0e4.83c8bf"]]},{"id":"120197bc.91df2","type":"schedex","z":"1fe83a7f.e0f0b6","name":"Reset","passthroughunhandled":false,"suspended":false,"lat":"","lon":"","ontime":"0:00","ontopic":"","onpayload":"","onoffset":0,"onrandomoffset":0,"offtime":"","offtopic":"","offpayload":"","offoffset":0,"offrandomoffset":0,"mon":true,"tue":true,"wed":true,"thu":true,"fri":true,"sat":true,"sun":true,"x":530,"y":4300,"wires":[["e3aaae46.d1ab3"]]},{"id":"54b61769.9755f","type":"api-render-template","z":"1fe83a7f.e0f0b6","name":"Subtract 1","server":"63517380.eb951c","template":"{% if states('sensor.mail_amazon_packages_delivering') | int > 0 %}\n  {{ states('sensor.mail_amazon_packages_delivering') | int - 1 }}\n{% else %}\n  0\n{% endif %}","resultsLocation":"payload","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":550,"y":4380,"wires":[["c4aae13.3acb2a"]]},{"id":"c4aae13.3acb2a","type":"change","z":"1fe83a7f.e0f0b6","name":"Set to 0","rules":[{"t":"change","p":"payload","pt":"msg","from":"","fromt":"str","to":"0","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":4380,"wires":[["4b7d0e4.83c8bf"]]},{"id":"2bd4738b.515f9c","type":"api-render-template","z":"1fe83a7f.e0f0b6","name":"Subtract 1","server":"63517380.eb951c","template":"{% if states('sensor.mail_amazon_packages_nearby') | int > 0 %}\n  {{ states('sensor.mail_amazon_packages_nearby') | int - 1 }}\n{% else %}\n  0\n{% endif %}","resultsLocation":"payload","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":550,"y":4220,"wires":[["28e5988a.c3dda"]]},{"id":"28e5988a.c3dda","type":"change","z":"1fe83a7f.e0f0b6","name":"Set to 0","rules":[{"t":"change","p":"payload","pt":"msg","from":"","fromt":"str","to":"0","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":4220,"wires":[["33f49d42.466b7a"]]},{"id":"460d5c63.ba0554","type":"ha-webhook","z":"1fe83a7f.e0f0b6","name":"Tasker","server":"63517380.eb951c","outputs":1,"webhookId":"","payloadLocation":"payload","payloadLocationType":"msg","headersLocation":"","headersLocationType":"none","x":230,"y":4300,"wires":[["a76f2d8.99415d"]]},{"id":"63517380.eb951c","type":"server","z":"","name":"Home Assistant","legacy":false,"rejectUnauthorizedCerts":false,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

Conclusion

This is definitely a niche thing but I know some people out there like their packages tracked and so far over the course of a week this has been rock solid for me. I hope you find it useful!

Object Detection with BlueIris and Deepstack

Wed, 26 Aug 2020 18:59:08 +0000

NOTE: See revised post here. Below is being left for posterity, but is not longer what I would recommend.

As I previously mentioned, I recently ditched my Wyze cams and moved over to Eufy cams for indoor. I also decided to buy some outdoor cams to mount and watch my front door and driveway in real time. However, I got tired of getting lots of false alerts due to trees/bushes moving in the wind, bugs flying by the camera at night, etc. So I started to explore ways of weeding out the false positive alerts and only getting alerts when there actually was a person at my door or a car in my driveway. Went through a few set ups but I’ve found one that has been working terrifically for a few days so wanted to share.

What You’ll Need:

Video Camera - Inside I use my Eufy cams, and outside I picked up these Reolink RLC-410W’s. I wanted to do POE but due to house design issues I couldn’t run the necessary wiring. So far the 410W’s are working great.

NVR Software - I use BlueIris, but really any NVR software should be able to work, so long as you can use it to trigger the AI software to analyze the camera snapshot.

AI Software - There’s a LOT of options out there, some of which are local and some of which rely on various cloud setups. Personally I wanted to only use local processing. I know several people who use the Coral USB Accelerator and Frigate (which uses TensorFlow). I personally settled on using Deepstack which I have found to be fairly accurate and when configured correctly not overkill on my server. Though I’m sure if I figured out a way of adding some kind of Coral-like device the processing would get significantly faster.

HomeAssistant - This really should go without saying at this point…

Approach #1

Initially, I was inspired for this project by this video by Rob over at TheHookUp.

In a nutshell, his set up using BlueIris to take a snapshot when motion is detected on the lower definition substream of his camera, which is then processed by Deepstack to determine if there’s a person. If there is, it triggers recording in full 4k resolution. The goal for Rob was to limit the recording in 4k since it would chew up a lot of HDD space quickly to record in 4k 24/7.

I followed this approach at first and was very impressed at how simple it was. And it worked. However, I quickly ran into a limitation for my own use case. I wasn’t concerned with my cameras all recording at full quality (the Eufy ones don’t have a substream) but I wanted a snapshot sent to my phone when the camera detected a person (or a car) at the front door or in the driveway.

What I had set up based on the above approach was 1) BlueIris would detect motion and take a snapshot, 2) the AITools application would send the snapshot over to Deepstack, 3) if Deepstack indicated it was a person/car, AITools would then trigger the camera, which 4) would send an MQTT alert to HomeAssistant to take a snapshot of the stream. The result was, as you might expect, that often times the person would no longer be in the frame by the time HomeAssistant took a snapshot due to the lag of DeepStack processing (even though it was only a few seconds). So that forced me back to the drawing board.

Approach #2

When I would check the log in AITools I would see that Deepstack was pretty accurately detecting that there was a person/car. The only other way to get images out of AITools was via Telegram. But the downside here was that there was no way to filter out snapshots when I didn’t want them (ie when the front door is opened by me to go get the mail). Plus it put a decent amount more stress on my CPU since it would send a LOT of images to Deepstack to be processed.

So basically what I wanted was a way to have BlueIris detection motion, send a trigger to HomeAssistant, which would then (depending on certain conditions I might want to set) take a snapshot and send the snapshot to Deepstack, which would then return the same image if it detected a person/car. So no more middleman and trying to get the images synced up.

To do this, I found this custom component. There’s also one that can detect faces but I have not given that a go. The instructions on the GitHub page for installing Deepstack via Docker and getting everything set up are pretty well done. Just a few comments/things I’ve discovered:

There’s a few different versions of the Deepstack container. The GitHub page examples all use the noavx tag, versus the latest tag supports AVX. You can determine if your computer supports AVX with grep avx /proc/cpuinfo in Linux. HOWEVER, I couldn’t get the “latest” image to work properly as activation seemed broken. The noavx image works fine so that’s what I went with. There’s also cpu-x3-beta or gpu-x3-beta which supposedly include a number of improvements, but based on reports I’ve seen the processing is currently a LOT slower. Deepstack is in the process of open sourcing the software so hopefully when that process is complete we’ll see improved versions.
I would also highly suggest playing around with the Deepstack UI built by Rob Markcole, which you can find here. It will help you set up the entities in HomeAssistant since you can see how adjusting the x/y coordinates will affect image processing.
If you’re running BlueIris, then I would suggest checking out the custom component here. You can get might higher quality streams into HomeAssistant this way (over the MJPEG one I had set up), and you can easily get various alerts set up (without having to define them all in YAML). Note for the alerts you’ll want to refer to the BlueIris manual located here. This also lets you use BlueIris’s profiles, which was something I hadn’t used too much before but absolutely am now to help control when alerts are sent to HomeAssistant and thus limiting the amount of logic in HASS/NodeRed that’s required.
I think Rob mentions it in his video, but I would encourage you to set the motion sensitivity of the cameras to pretty high, since any false positive alerts should be sifted out by Deepstack.

Conclusion

Not a lot of “how to” in this post since I think the documentation is pretty well explained. I intended this more as an explanation of how easy it is to set up AI-based object detection in your home and boost the accuracy of your motion alerts.

Overall, this was a pretty easy project to set up. It took some tweaking of the motion sensitivity of the camera and getting all the alerts set up but overall the result is that I no longer get false positives of “someone at the door” when there really isn’t.

Eufy Indoor 2K (including Pan/Tilt) Camera Review

Fri, 03 Jul 2020 18:17:12 +0000

As I noted in one of my first posts, I have used the Wyze V2 indoor cams for almost 2 years. I started off with the cameras flashed to the custom Dafang-Hacks firmware which worked OK. I was using Blue Iris as my NVR and I saw a lot of disconnects/breaks in the recording. Eventually Wyze released a custom firmware version that supported RTSP. However, after more than a year development on the firmware has gone no where. I still saw tons of drops/disconnects while recording and the picture quality was just ehh. I know there are many people out there who have had lots of success with the RTSP firmware, but from what I’ve seen usually it’s people running only 1 or 2 cameras. I was running 6 and even after expanding my WiFi network capability by adding a dedicated wireless access point I still had drops. I had more luck using MotionEye to record but then there was no sound.

To be clear, the Wyze Cam v2 was remarkable. It was the first decent camera that sold for $20 ($25 with shipping) and I had no issue recommending them to friends and family looking for a cheap way to get cameras in their house. They even could be mounted outside and fared fairly well all things considered. The streaming quality was pretty decent for the price and the free cloud storage (with the 12 second clips) supplemented by an SD card was pretty awesome. But after 2 years the limitations (and no progress on the RTSP end or getting the camera integrated into HomeAssistant) really was getting to me.

Then in March I saw an article about Eufy releasing an indoor 2K pan and tilt and a stationary camera. The retail prices were announced at $49 and $39 respectively, but pre-orders saw about a 25% discount bringing the prices down to pretty much the same prices as the WyzeCam v2 and WyzeCam Pan. I know and like the Eufy brand after being very pleased with my doorbell cam so I was interested in giving them a go. So I went a bit crazy and ordered enough to replace every Wyzecam in my house.

General Features/Thoughts

Let’s start with the obvious question: did the RTSP work better on the Eufy cam? YES. A million times yes. I’ve had all 6 cameras streaming to Blue Iris without a single drop (or at least one that’s noticeable) for almost 2 weeks. That alone makes the camera worth it. In terms of picture quality, it’s mostly a tie with maybe a small nod towards the Eufy. I definitely don’t think it’s “2K” in terms of quality but the color temps feel a bit more normal and it’s easier to make out facial features.

Other vast improvements I’ve noticed over the Wyze cam are 1) the app/stream is very quick to load and rarely fails, 2) the speaker quality for the two way talk is pretty decent, and 3) more importantly the microphone is significantly better on the Eufy cam than the Wyze. Another benefit of the Eufy cams is the on-board AI that is able to detect persons, pets and crying. Wyze introduced person detection (though was forced to pull it when the company they used got bought by Apple) but from what I’ve gathered it was just OK. The Eufy detection is pretty much spot on, though occasionally I get a notification about a pet being detected when my son is playing with his stuffed animals. I’m sure that will get better with time.

Some other things I’ve noticed that differentiate the cameras. First, an SD card is required in order to receive notifications on the Eufy. I was planning on using an SD card anyway so this wasn’t a big deal. I also appreciated the ability to rate-limit the notifications so I wouldn’t get constant notifications when motion was detected (anywhere from no cooldown to 5 minutes). Pairing the Eufy cam to my wifi network was a much simpler process than with the Wyze. Sometimes I would have issues getting a Wyze cam connected but I had no issues with the Eufy. Perhaps because the Eufy pairing process uses bluetooth instead of a QR code the camera scans.

Recording is also very easy to set up and you can simultaneously record to an SD card as well as to an NVR via RTSP (I do this because sometimes it’s easier to find the clip I want on the camera without having to go into my NVR software). Video scrubbing on the Eufy app is infinitely superior to the Wyze experience. On the Wyze I’d so often be unable to find the spot I needed because it would take forever to load the footage (and oftentimes would just flat out fail to load). So far no issues on the Eufy, and as an added bonus there’s a clear way to tell when the camera has detected motion/sound (yellow bar as opposed to the general blue).

Indoor 2K Camera

I just wanted to include a few thoughts that are particular to each camera as there are some slight differences to note. The Eufy camera is slightly bigger than the Wyze thanks to the larger stand that doesn’t compact like the Wyze stand can. But that’s not a huge deal to me. The stand also isn’t as adjustable it’s flexible enough that I don’t think it’ll be a big problem. Unlike the Wyze, the mount is not magnetic so if you want to mount the Eufy cam on a wall you’ll need to use the included mounting plate with screws. Somewhat limiting but also not a huge deal.

Pan/Tilt Camera

I don’t actually have a Wyze Pan to compare the Eufy one too but I don’t have a ton to say about the Eufy pan that hasn’t been said above. There’s a standard mounting plate in the box that lets you screw into most available wall mounting solutions which is nice. The camera motor is virtually silent (my kid never reacts when I move the camera around at night even before he falls asleep), versus I’ve read some complaints about the Wyze Pan motor being a bit loud. Another cool feature of the Eufy is that when the camera is off the lens rotates back into the body (which also exposes the SD card slot, btw) so it’s easy to tell that the camera is off.

Conclusion

Overall I love the Eufy cams and I don’t see myself going back to the Wyze cams unless the V3 is something remarkable (no, there’s no official word on if/when a V3 is coming). I loved the Wyze cams but they’re long in the tooth at this point and the hardware limitations are obvious at this point. The fact that it can’t handle RTSP but the Eufy can is I think indicative of this issue. My gripes with the Eufy are pretty small (no truly working HomeAssistant integration, limited ability to control the notification schedule) but other than that they’re great and even for slightly more expensive than the Wyze cam at $15 I think it’s money well spent. Though I would expect there to be frequent discounts on the cameras in true Anker/Eufy style once they get past the initial roll out).

Adventures in Individually Addressable LEDs

Fri, 19 Jun 2020 19:28:58 +0000

So as I mentioned in my previous post about desk detection, I recently redid my office and installed some cabinets. The lighting was OK but I really wanted some extra light (and at times, pizazz!) and had seen a lot of people talking about individually addressable LEDs. So I decided to start off with a small project and see how it went. I also was tired of dealing with my outside solar-powered pathway lights that only worked half the time and just looked eh so decided to install some LED strips outside.

I learned a LOT from both projects and interestingly though I tried to wire them the same they both ended up with different set ups because I used different software on the ESP boards. This one is going to be quite long and have a lot of pictures/diagrams/links to videos.

Introduction

Ok, so let’s start with the obvious question. What is an individually addressable LED? There are two kinds of LEDs out there. The first is the ones you see with a remote that can change to lots of colors, but the entire strip is a single color. Individually addressable are, surprise, strips where each individual LED can be a different color or on/off. This is what you see when you see light shows like this on peoples house for the holidays:

Now I’m not saying setting up a show like that is easy (that’s a whole other category than what I’m doing here). For most home automation projects you’ll want individually addressable LEDs. They’re cheap, easy to use (once you figure it out), and can be controlled by an ESP8266/ESP32. Plus, they’re lots of fun!

Hardware

There are a lot of options when it comes to individually addressable LEDs. Dr. Zzs does a terrific breakdown of the different types and the things you should consider. It’s geared more towards an outdoor LED display (and goes into more than just the hardware, we’ll get to the rest of it below) but there’s some good info on the LEDs themselves in here:

You may have already seen this next one because Dr. Zzs mentions/links to it in his video but here’s a great rundown of the different varieties of LEDs (warning, it’s somewhat technical but makes for a great watch):

And just one more for good measure that’s also holiday-ish focused but has a lot of useful info:

In summary, here are the things to consider:

Strings versus strips2) 5v versus 12v3) Power supply4) If strips, what kind of waterproofing (personal suggestion is <IP30 for anything inside and IP65 for outside (IP67 is kinda overkill and harder to work with).

For my project I ended up using WS2812B 5v strips. I did end up getting a set of 5v bulbs, but we’ll get to why in a bit. Here’s the rest of what you’ll need

ESP8266 NodeMCU (3-pack) - $13.99 (Assuming you don’t already have from a previous project)
Electrical wire - Anywhere from 18-22 gauge wire is fine. The higher gauge/thinner wire (like 22 gauge) works for small projects. If you’re doing a large or outdoor project, get lower/thicker gauge wire (like 18 gauge)
Soldering iron - $15.99 (optional depending on how you chose to resolve the problem I’ll explain below with data wire corruption).
Jumper cables - $6.98 (this should be plenty for several projects and gives you options for male-male, female-female, and male-female connectors)
LEDs! I’m not going to make any particular recommendations here since there’s a ton. Just search on eBay, Amazon, or AliExpress and you’ll find a ton. I’ve bought Chinly, Alitove, and BTF-Lighting and all are good.
Power Supply - As Dr. Zzs explained, the size of your power supply will be dictated by how many LEDs you want to power. His “Zzimple rule” of 50 LEDs per amp is decent though know you want to do white light at higher brightness, budget for more amps (maybe more like 30 LEDs per amp).

For my outdoor project, I added the following:

Waterproof outdoor box - $22.99. I like this one, though it’s HUGE. There are smaller sizes (I got the large) so get whatever size you need.
Aluminum Cord channel - ~$35.00 for 10m of channel. You pick what you like best. Just be aware that there are differing levels of “diffusion” (ie how hard is it to see the individual LEDs). How strong a diffuser you want will depend on your use case and personal preferences. I like the Muzata ones I got, you find what works for you.

Software

There are two popular options for controlling LEDs: ESP Home and WLED. Both require flashing the ESP board with custom firmware (ESP Home is a bit easier here but WLED can use the ESP flasher software so it’s not that much of an advantage and both integrate very painlessly as integrations in HomeAssistant (as simple as entering the IP address of the ESP board) so don’t let that drive your decision. I’ve used both and here’s my 2 cents.

ESP Home

ESP Home is absolutely fantastic for doing a LOT of things with ESP boards, as I’ve covered before. It’s strength is its flexibility and simplicity. But that’s also a drawback for LEDs, depending on your use case. If you want a lot of fun light effects, then while ESP Home has a number of effects, they aren’t quite as much fun as WLED’s without some effort. But if all you’re looking for is a simple LED setup I think ESP Home is perfect. For my office setup it was really great because I got to use the same ESP board that I use for detecting brightness. If I’d wanted to use WLED I would have needed 2 separate ESP boards since I wanted individual strip control and WLED can only use a single data pin (D4). More on this below (sorry for being a tease).

I suggest using the FastLED library though some people claim better results with the NeoPixelBus library. Both worked fine for me so pick whichever you prefer.

One other tip, you can set up “light partitions” which essentially are different segments of lights. That took me a bit to find but was very useful.

WLED

WLED is brilliant. Absolutely brilliant. There are SO MANY EFFECTS it’s hard to find which one is my favorite. WLED has an Android app that’s very easy to use and WLED allows syncing of the controller with other WLED controllers so you can have the same effects running on different strips, all controlled from one spot. Handy for light shows. You also can sync with Alexa and Hue natively which is also useful for some (not for me).

The main drawback to WLED IMO is that you can only use a single data pin (D4) without recompliling the code yourself (at least that’s my understanding, please someone correct me if I’m wrong!). It also has so many bells/whistles that for a simple LED strip it’s kinda overkill.

Which one?

Again, I think both have their use cases. Want a fun light show with cool effects to impress your friends? Go with WLED. Looking for simplicity? ESP Home is the way to go.

Wiring it Up

This is definitely the most complicated part and the part I struggled the most with. Maybe you’re better at understanding electrical wiring than I am but it just took me a lot of time to wrap my head around some of the concepts/how to wire things.

There’s 3 wires you need to deal with. 5 volts (usually red), ground (usually white) and data (I see it normally as green). Below is a wiring diagram for a basic setup.

Basic LED wiring setup

A few notes here. First, DO NOT TRY TO POWER YOUR LEDS OFF OF USB POWER TO THE ESP BOARD. You will let out the magic smoke and will have a dead board (ask me how I know…). You can power the ESP board off your power supply by connecting it the 5v wire to the VIN pin on the ESP and ground to a ground pin on the ESP. Second, I have the data wire running off D4. Remember for WLED that’s a requirement, but for ESP Home you aren’t restricted to just D4 (there’s a few limitations but a few of them will work). Third, you may have noticed the “<2 feet” comment next to the data wire. This is because if your data wire is too long (basically 2ish feet on 5v), the data signal can get corrupted and the result will be flickering/screwy LEDs.

The Hook Up has a pretty good technical explanation of what the issue is and a solution using a logic level shifter (jump to about 1:30 into the video):

Here’s a wiring diagram:

LEDs with a logic level shifter

As explained in the video, the logic level shifter boosts the output of the data wire to 5 volts. To do that, you need to take a 3.3v wire and ground, along with the data wire, and feed it into one end of the logic level shifter. The output then is 5v that you feed back to the main line and then the data wire goes off and can be very long (my outside wire run is ~25 feet without issues).

As an FYI in case you’re splitting your data wire, I found it was best to split the wire BEFORE you feed into the logic level shifter. I initially had mine done afterwards and i ran into problems. This was the board I used. Your results may vary.

For my indoor set of LEDs I really didn’t want to have to solder the pins to the logic level shifter board so I took a different path based on a Dr. ZZs’s video. Basically instead of a logic level shifter, if you use an LED wired near the ESP board it acts very similar to the logic level shifter in boosting the voltage on the data wire. Jump to 11:00 in the video for an explanation/visual demo.

I’m a visual person so again here’s a wiring diagram for you to look at.

Wiring with a sacrificial pixel

Note that WLED allows you to turn off the first LED in settings (probably for this reason). For ESP Home I set up a light partition to handle.

Also, to stress using the proper wire gauge, don’t be like me. Otherwise you end up with this:

Oops.

Results

Here’s my final result for the outside project:

Yes the wiring in the box is ugly (and I can probably cut it) but it works so I’m leaving it alone

And inside:

Here’s an example of the wiring with sacrificial pixels

Conclusion

These were not easy projects for me, but ultimately I learned a LOT. I’m debating expanding the outdoor LED setup for a holiday display but not sure I want to take on that hassle/headache. I do think my next project is going to be under the bed LEDs that come on when the SO or I get out of bed at night (turn on dim red for night vision). I mean, I already have the ESP board just sitting there waiting….hahaha.

Hope this is useful to some who are considering taking on an LED project of their own.

Desk Detection with ESP8266's

Sat, 13 Jun 2020 18:31:21 +0000

This is a bit of a tease since part of this project was to install individually addressable LED strips (WS2812B’s) under the cabinets I recently installed in the office and I’m waiting to write that part up until I finish a more complicated outside install. But I still think it’s useful to discuss how I’m detecting whether I am at my desk to turn on the LEDs or not.

Since I didn’t want the LEDs on all the time unless I was sitting at my desk I needed to figure out a way to detect whether I was sitting at my desk or not.

Attempt #1

For my first attempt, I decided to try the capacitive sensors I’d been using for bed detection. Those worked….OK. I started with 2 and eventually made 4 (actually 8 total as I made 4 for my spouse’s desk as well). Benefit here was that I already had a spare ESP32 lying around to use (and there are 8 touch-enabled pins on the ESP32) and making the sensor mats isn’t incredibly difficult so I really didn’t have much to lose by trying.

But no matter how much I tweaked them (changing placement, adjusting the sensitivity, etc) I couldn’t get them to work reliably. Sometimes I got false positives when the chair was in a certain spot, other times I’d be sitting/not moving but the sensor would turn off anyway. After a few days I decided I needed to take another approach.

Brainstorming Other Ideas

Did some searching and found 2 other solutions that people had tried. The first is to use a contact sensor (like a door sensor). The idea is that when you sit in the chair you’d push the sensors close enough to each other that the sensor would turn on. I didn’t like this solution because neither my spouse’s desk chair nor mine would have made this easy due to the design. Also, neither one has a lot of up/down if we’re seated or not so it didn’t seem like it would work well.

The other solution, and the one I finally settled on, was to use an HC-SR04 ultrasonic sensor, like this:

This little guy sends out an ultrasonic ping at a pre-defined period (potentially every few miliseconds if you really wanted) and detects how far away an object is from it (I think it has a range limit of about 2 meters).

Setting Things Up

So the setup here is really easy. You’ll need the following (and may already have some of the parts around if you’ve done any other ESP-based projects):

An ESP8266 or ESP32
An ultrasonic sensor. I used the HC-SR04. They’re fairly cheap (about $7-9 on Amazon for 5 can be easily found).
Jumper wires
Power adapter/micro usb cable to power the ESP.

That’s it! I used ESP Home for this and followed the instructions on the site to wire it. It’s pretty simple. VCC to VIN (5 volts), ground to ground, and then TRIG/ECHO to a data/GPIO pin of your choice (I used D5 and D6).

Turning to the code, here’s my config:

sensor:
  - platform: ultrasonic
    trigger_pin: D6
    echo_pin: D5
    name: "Ultrasonic Sensor"
    update_interval: 1s
    id: desk
    filters:
      - delta: 0.1

binary_sensor:
  - platform: template
    name: "Desk Sensor"
    lambda: |-
      if (id(desk).state < 0.75 ) {
        return true;
      } else {
        return false;
      }
    filters:
      - delayed_on: 2s
      - delayed_off: 5s

The actual sensor is what measures the distance. I have it update every 1 second which I find to be fast enough for me. I also filter out small changes (anything less than 0.1m in change will not be reported to HomeAssistant, that’s what the deta line is for). I also created a binary sensor to report on/off depending if the distance is below a certain level. For me, I found 0.75 was fairly accurate though I still may need to tweak a bit. I further filtered this to delay the “on” by 2 seconds to just make sure there wasn’t a whacky one-off reading and similarly 5 second delayed off to make weed out any small movements I might make.

I then added the sensors to HomeAssistant via the ESP Home integration (supppppper easy) and then created some automations to turn the lights on/off.

For the installation I just used some mounting putty and electrical wire to run the USB cable around the edge of the desk. Finished result is that you can’t even really see it.

Not the prettiest install but it works

Ease of project: 9/10. The first iteration with the under-mat sensors was a lot more complicated (requiring soldering/long wire runs and making the mats). The ultrasonic sensor by contrast was incredibly easy and simple to do. Probably the easiest ESP project to date.

Next up is either a summary of my experiences with individually addressable LEDs and ESP boards (with both WLED and ESP Home) or a write up on my new Eufy indoor cams. But since I’m still waiting on my order of the pan/tilt cams I may delay doing a review of the Eufy cams until I have both (brief review: it’s muuuuuuch better than the Wyze cam in a lot of areas and equal in all the rest with no drawbacks).

Setting up NodeRed

Fri, 05 Jun 2020 00:34:40 +0000

As mentioned in one of my first posts, I love NodeRed and use it for virtually all of my automations. It’s a visual automation editor that I find much easier to use than the YAML-based automation system in HomeAssistant. If you’re a visual person like me (and even if you aren’t) you may find NodeRed simpler to use. I also much prefer the easy debugging my automations that was difficult to do in HomeAssistant.

In this post I’m going to go through how to install it, install some basic palettes, and how the integration with HomeAssistant works. Let’s get to it.

Installation

Docker

Installing NodeRed is really easy. I run it in Docker and below is my Docker-Compose file that you can use. If you aren’t already using Docker or Docker-Compose I’ll point you here.

version: '2'
services:
  nodered:
    container_name: nodered
    image: nodered/node-red
    volumes:
      - ./nodered:/data
      - /etc/localtime:/etc/localtime:ro
    restart: always
    ports:
      - "1880:1880"
    environment:
      - TZ=America/New_York

That’s all you need. Run docker-compose up -d nodered (while you’re in the directory where the docker-compose.yaml file is located) and then open your browser. Type in http://YOUR_IP_ADDRESS:1880 and you should be greeted with a page that looks like this:

Your new NodeRed install

If you aren’t, a possible issue is ownership of the NodeRed folder. Run sudo chown -R YOUR_USER:YOUR_GROUP /path/to/your/nodred/dir. If you don’t know what I mean by user/group see this article. Restart the container and you should be all set.

NPM Install

Don’t want to use Docker? That’s fine. You’ll just need to follow the instructions over on the NodeRed website here. In a nutshell, you’ll need to install node.js and then run this command:

sudo npm install -g --unsafe-perm node-red

Then all you need to do is run node-red from terminal and NodeRed will start. You’ll probably want to run it in screen or create some kind of systemd process to run NodeRed in the background as you want it running all the time (otherwise no automations!).

Home Assistant Add-On

It’s also possible to install NodeRed if you’re running “Home Assistant” (formerly known as “Hass.io”). Check out more information here.

Next Steps

So now we have NodeRed installed and running. Let’s figure out what everything is. If you’re a visual person, check out this video by The Hook Up (2 years old but still pretty accurate though some things have changed).

On the left side of the screen are all the different “node” types you can use. A node is a thing that you can do to execute a function/command depending. You’ll see more on this later

At the top you have something called “Flow 1.” Each “page” in NodeRed is called a “flow,” but personally I think it’s easier to just think of them as pages.

On the right side is an info bar and also very importantly a debug panel button and the deploy button. The info bar is useful when you click on a node to get some information about how the node can be used/what features it has. The debug panel (bug icon next to the “i” is useful for debugging your automations and seeing where you have problems. Finally the “deploy” button is what “saves” everything. You can configure the deploy type by clicking the arrow next to “deploy.” I would suggest switching the deploy style to “Modified Nodes.” The default of “Full” will literally redeploy every single node in NodeRed. So if you’re running a timer, it will reset the node every time you hit deploy. Instead, if you do “modified nodes” it will only deploy the nodes you actually changed. This makes more sense to me, but the decision is yours.

Useful palettes

There are TONS of palettes out there that let you do all sorts of crazy things. To install a node, all you need to do is click the 3 lines in the top right corner and then “Manage palette.” Click on “Install” and from there you can search for an install a module/palette.

Personally I’ve found a few very useful. Here’s a list with a brief description of what they do and why I like it.

node-red-contrib-home-assistant-websocket - We’ll get to this one more in a minute, but this is what you need to connect HomeAssistant and NodeRed.

node-red-contrib-schedex - This node acts similar to a time trigger in HomeAssistant but also lets you add an off time as well as use sunrise/sunset as a time (or golden hour, dawn/dusk, and a whole host of other times of day) along with offsets if you need to adjust the time slightly. I use this one a lot to run automations at certain times of day/days of the week.

node-red-contrib-time-range-switch - Time Range is similar to a time or sun condition in HomeAssistant. When an input is passed in it checks to see if the time of day is within the parameters you set (such as between 12pm and 3pm, or sunrise to sunset). If yes, you can set the output to go one way. If no, then the output can be sent another way.

node-red-contrib-weekday - Similar to Time Range but this one is for days of the week. If an input comes in and the day is one that you select (say Monday-Friday), the output can be routed one way, and if it’s outside of that it can be routed another way.

What about node-red-contrib-bigtimer? I know a lot of people swear by BigTimer. Personally I found it way too cumbersome and complicated for my purposes. There’s so many outputs/things you can do that I was just overwhelmed and felt the 3 time/date palettes above were much easier to work with/use.

HomeAssistant

Let’s get NodeRed talking to HomeAssistant. Go to Manage Palettes and install, and search for node-red-contrib-home-assistant-websocket. Then click “install.” A popup box will show up asking you to read the documentation. Just click “install” again. You should see a notification that a bunch of new nodes were added to NodeRed. If you scroll all the way to the bottom on the left side, you should now see something like this:

Before we get into what each node does, we need to configure NodeRed to talk to HomeAssistant. Pick any HomeAssistant node, click and drag it onto the grid. Then double click on the node and you’ll see a screen that looks like this:

Click the pencil button next to “Add new server.” If you’re running HomeAssistant/HASS.IO then click the “I use the Home Assistant Add-on” option. Otherwise fill out the base URL (I’d suggest using your local server IP/port even if you have have HomeAssistant publicly accessible). You also will need to get a Long Lived Access Token (see below image). Once you have that, click “Add” and you’re set.

Scroll all the way to the bottom of your profile in HomeAssistant and click ‘Create Token.’ Name it, then copy the value to the ‘Access Token’ line in NodeRed

Now that we have HASS and NodeRed talking, let’s go through some of the nodes. Personally I don’t use the Webhook, Fire Event, Get Entities, Get History or API nodes. I’m sure they have use but I haven’t found a need for them. Here’s a quick use case for the other nodes.

Events: all -> Useful for picking up on events like deconz, zwave.node_event, and others. I don’t use this one a ton but it’s super useful when you need it. Only has an output.

Events: state -> Probably one of my most used nodes. You can pick an entity_id to target, or you can do a substring or regex to use more than one entity at a time. You also can make the node act like a true/false if you put a state, OR you can leave the “if state” field blank and it will output whenever the state changes. This is the man way I trigger my automations in NodeRed.

Trigger: state -> Very similar to the Events: state node, but has the added benefit that you can set constraints on it. For example, if you have a temperature sensor and you only want it to trigger when the temperature is between 70 and 72 degrees just set the constraints appropriately it will output only when those conditions are met. You also can add additional constraints for other entities. So if you want the automation to trigger, but only when the temperature sensor is between 70 and 72 AND you aren’t home, you can do that with the trigger: state node. You also can set different outputs depending on the state

Poll state -> useful if you want to check the state of an entity and trigger if the value changes. I don’t find a ton of use for this since most of my sensors update regularly but if you have some that are slow to update, it can be useful.

Call service -> the main way to do something with HomeAssistant. This will probably be the node at the end of your automation. It’s basically the same thing as the “action” in HomeAssistant YAML automations, so think of it that way.

Current state -> I use this mostly like the conditions in HomeAssistant YAML automations. You can use it to check the value of an entity and to send the flow one way if it’s true and another way if it’s false.

Get Template -> I often use this node similarly to the current state node. Basically you can use it to run a template in HomeAssistant and use that output of that template in your automation. I like to use template nodes to replace strings of current state nodes as you can do {{ state('input_boolean.number1') == 'on' and state('input_boolean.number2') == 'on' instead of doing 2 different current state nodes. One thing I do with template nodes is I usually change the results output to msg.results. That way I can keep the value of the msg.payload but then also have the output from the template in case I need to use that later. Also, if you’re using the template node to evaluate conditions of entities so the result is true/false, you’ll want to put a switch node after it so you can filter out if the statement is true/false depending on which you want. The node will always output no matter what.

Wait until -> A handy one to use if you want to trigger an automation but then wait a bit before continuing. Example I have is that when the house door to my garage is opened it sets off an automation but then waits 5 minutes to see if my car starts. If it does, then the automation continues and opens the garage door. If it doesn’t then the automation stops. You can customize how long for the timeout (0 is no timeout).

NodeRed Integration

There is a NodeRed Custom Component for HomeAssistant that provides a few more features, most notably the ability to turn some things on/off in NodeRed from HomeAssistant as well as create sensors in HomeAssistant. This has some interesting use cases that I’m still exploring so I would encourage you to install and take a look. You can read more about it in the documentation here.

Other Useful Nodes

There’s a lot of other really useful nodes in NodeRed that I use all the time. The debug and inject nodes are absolutely invaluable for testing out new automations to make sure they work without having to wait for the automations to trigger on their own. The link in/out nodes are extremely useful for reusing bits of automation or service calls without having to repeat them over and over again. And the switch and change nodes are equally useful in causing my automations to split/move around in different ways depending on the value that’s received.

Conclusion

The above really is just scratching the surface of what NodeRed can do. I’m a visual person so I find it much easier to use that the HomeAssistant automation system, though I’m glad I learned that as it does help with figuring out service calls for NodeRed. I think the main thing I really like with NodeRed is the ease I can reuse things without having to write out hundreds of lines of YAML. I would strongly encourage you to try out NodeRed. You can easily set it up and use it in parallel with the HomeAssistant automations as you slowly transition things over. So you have nothing to lose!

MergerFS + SnapRAID - Perfection?

Tue, 02 Jun 2020 01:09:20 +0000

Recently I’ve begun listening to the Self-Hosted podcast. One of the hosts, Alex, is a huge fan of MergerFS. He’s written about it a few times as the “Perfect Media Server” solution. You can here him talk about it here or read one of his write ups here here and here. For the record he does also really like ZFS, but his write up of MergerFS and SnapRAID intrigued me for 2 reasons:

The ability to increase the size of the data array easily after you’ve already created it.
The ability to use drives of different sizes.

ZFS/MergerFS

Originally I was using ZFS and it was….fine. I created my drive pool using Raid-Z (3 drives, 1 parity disk) and mounted it and….never really gave it a second thought. I know I wasn’t using a lot of the more fun features of ZFS but that really wasn’t what I was after, I just wanted the ability to pool drives into an array and have a parity disk for “backup” (RAID IS NOT BACKUP but it’s a nice security blanket).

But I slowly was reaching max capacity on my array and realized I was going to have trouble expanding the array. And by trouble I meant “uhhhh….”. ZFS pool expansion has been on the “coming soon” list since 2017 but I haven’t seen any progress. One solution would be to create a new array and either add it to my server (which I don’t have room for) or transfer the data over (requiring another computer). Honestly neither option was super appealing to have to do on an even infrequent basis. Also, ZFS is complicated because you need to buy matching drives which adds to the cost/complexity.

So I decided to look into MergerFS and SnapRAID. The install is really easy (and well laid out in the above linked-to posts). Essentially, it’s 1) install MergerFS, 2) figure out the drive serial IDs, 3) create directories for the mount points (including one for the “usable” mount point, 4) edit fstab to mount the drives, and 5) run mount -a. That’s it. You can add and remove drives from the mount fairly easily as well so it’s really very flexible. Which is great for a media server because I can slowly add HDDs in or replace dead ones instead of being constrained by the limitations of ZFS.

SnapRAID

For parity, SnapRAID is pretty cool too. It’s not instantaneous parity like ZFS so technically if you have data loss between backups intervals you’re screwed. But here’s actually where another advantage of MergerFS comes into play. All the data for a file is stored on a single disk. So if your HomeMovies.mkv file is on Disk 1 and Disks 2, 3 and 4 fail….no worries, your home movies are fine. Unlike with something like ZFS where if you exceed the fault tolerance of the array (i.e. you’re running RAID-Z/RAID5 and lose 1 disk, or RAID-Z2 and lose 2 disks, etc) you lose ALL the data of the array. Pretty sweet.

Setting up SnapRAID was a bit more complicated than MergerFS because I had some issues initially with SnapRunner but those issues worked themselves out over time and now it’s all fine and dandy. Again, I used Alex’s guide to build SnapRAID and run SnapRunner so I would point you there for a how to.

Data Transfer

This is more of a “lulz” bit but because I was out of space in my server (no more SATA/power connectors) I needed a way to transfer data over. Recently I’d upgraded my desktop motherboard/CPU and I had it lying around still. So I did the drive swap and set up the spare “computer” and transferred the data over. Took a day or two to transfer the few TBs of data and then I retired the setup, but overall it went well and now I have all my data on the new array of disks and can expand it as needed without having to do this again, which is a relief.

Frankenserver

So that’s now how I’m doing my storage in my server. MergerFS+SnapRAID ftw. I would highly recommend it over ZFS for most people building a home server array.

ESPs ESPs everywhere!

Wed, 27 May 2020 16:47:00 +0000

Ohhhh ESP8266. How I have grown to love thee. For a long time I was resistant to using them because I didn’t know how to solder and it just seemed to complicated to me. But I stumbled across a project (linked below) to create bed detection for super cheap and I was intrigued enough to give it shot. Being stuck at home thanks to COVID-19 also helped me want to find some projects to keep me busy. I also started to mess around with Tuya-Convert and flashing some ESP8266-based smart plugs with Tasmota which made me realize how simple and easy working with ESP8266’s is.

My ESP8266 of choice is the NodeMCU. There are other options out there, like the popular Wemo D1 Mini. The Wemo D1 is a lot smaller than the NodeMCU but requires soldering the pins on. Being a solder newbie I wasn’t comfortable with this option so I stuck with the NodeMCU and so far have been very happy with them.

Parts

Because I find it useful to link out to the products (and include pricing), here’s what I ended up buying. All links are to Amazon (note that these are straight links with no commission or anything to me).

ESP8266 NodeMCU (3-pack) - $13.99
ESP32 NodeMCU (2-pack) - $14.99
22 gauge electrical wire - $15.39 (this should be plenty for several projects)
Soldering iron - $15.99 (yes I didn’t own a soldering iron before this. Bad nerd.)
Lux sensors (10 pack) - $8.99 (read more below on why these)
SW420 Vibration sensors (5 pack) - $5.99
Jumper cables - $6.98 (this should be plenty for several projects and gives you options for male-male, female-female, and male-female connectors)
Boxes (5 pack) - $8.99 (I looked a while for something that would work, this was the best I could find. Required some minor dremmel work to put holes in for the cords/sensors but they work great)

Setting up ESP Home

There’s a few options for flashing ESP8266/ESP32. As mentioned above, Tasmota is one. I like Tasmota for pre-built switches/plugs/outlets/other devices. But for custom built things I felt that I wanted something a little easier to work with. The other popular solution out there is ESP Home. Both work well with HomeAssistant, but I found ESP Home to be a bit easier (it can be set up as an integration and all you need is the IP address of the device) and the configuration is via YAML. The ESP Home website has a lot of useful information/configs for different sensors, including a pretty cool list of DIY projects.

Installing ESP Home is really easy. All you need to do is run pip3 install esphome. That’s it. I also would also suggest installing the dashboard with pip3 install tornado esptool. Then to pull up the dashboard, run esphome PATH_TO_ESPHOME_DIR dashboard (replace “PATH_TO_ESPHOME_DIR” with a path of your choosing. Mine is /home/fuzzy/esphome for example). Usually I like installing things in Docker, but because the first time you flash a device via a USB cable and my server is in the basement, I figured I’d forgo Docker this time.

If you install the dashboard, flashing new devices is as easy as clicking the plus icon on the right side, naming the device, selecting what kind of device, providing the WiFi information (SSID/password), and that’s it. If you don’t have the dashboard, it’s also pretty easy as there’s a “wizard” to walk you through the process (see the ESP Home docs for this as it’s pretty self-explanatory).

I’d suggest flashing a generic ESP Home install without any sensors. Once you do the first flash you can then update over the air (OTA) which I found far more convenient.

With ESP Home being set up, let’s jump into some simple projects!

Ambient Light Sensors

Light sensor box. Had to drill out an end for the light sensor to poke through but not the worst

I have a bunch of Dome ZWave motion sensors that are supposed to be able to detect light levels. However, only about 20% of them actually work reliably for the light detection, most are stuck at 0 lux. I’ve really enjoyed having motion sensors turn the lights on/off in rooms as I walk through, but there are some rooms that get enough light generally during the day that I don’t need the lights on. I’d seen things like the Bruh Multisensor that got me interested in this as an option.

Bruh’s video in the above link is a pretty good description of the parts involved. However, I had a hard time tracking down the exact parts he used because a bunch of the links 404’d. I also wasn’t a fan of the TEMT6000 because again, soldering newb. So I went looking. My first option looked like it would work. However, upon receiving it and trying to wire it up using the Bruh guide, I realized that the values I was getting out of the sensor didn’t make sense (pitch black and bright light were only about 2 lux different). Turns out that version only had Digital Out (DO) and what I needed was an option that supported Analog Out (AO). A bunch more searching and I found one that had an AO pin and was presoldered. Perfect.

Note: A limitation of this sensor is that it maxes out at 200 lux so if you want to detect light variations in really bright rooms this probably won’t work. For my purposes it’s fine.

You need the AO pin (circled in red)

Wiring up the sensor is as simple as taking some jumper wires, connecting ground to a ground pin on the NodeMCU, VCC to a 3.3v pin on the NodeMCU, and AO to AO on the NodeMCU. Below is a wiring guide for the NodeMCU.

Wire to the pins boxed in red

Once it’s wired up, time to configure the code in ESP Home. Power up the NodeMCU, edit your YAML file by adding the below code at the end of the file, click “Save,” and then “Upload.” Once it’s done flashing you should have a working light sensor.

sensor:
  - platform: adc
    pin: A0
    name: "Room Brightness"
    update_interval: 60s
    unit_of_measurement: lux
    filters:
      - lambda: |-
          return 200-(x / 10000.0) * 2000000.0;

Change the name to match whatever you want.

You can adjust the update_interval to be faster or slower. For me, I found 60 seconds to be fast enough but adjust to whatever suits your purposes. You also technically can manually adjust the sensitivity with the screw on the sensor board, but I didn’t find this made a whole lot of difference so I left it alone.

Adding the sensor as mentioned above is as simple as going to the Integrations page in HomeAssistant, clicking the plus/add button, searching for ESP Home, entering the IP address and a sensor will be automatically added to HomeAssistant. You don’t really need a box/container but I found it nice to make it look “clean” and hide the LED lights.

Ease of project: 9/10

Only difficulty with this one was tracking down the right sensor and drilling the holes in the boxes for the sensor to poke through. Other than that it doesn’t get much simpler.

Dryer Vibration Sensor

I’ve always wanted some kind of laundry automation to tell me when the washer/dryer are done so I remember to move the wash to the dryer and fold the clothes in the dryer. My issue was that I have an electric dryer meaning I couldn’t just use a normal smart plug with energy monitoring. I looked into energy detection using clamps like this but at $50+ I didn’t think it was worth it. I also tried the Xiaomi Aqara vibration sensor but I couldn’t get it to reliable detect vibration when the dryer was running so gave up on that approach.

Then I stumbled across this solution which uses an ESP8266 and an SW-420 vibration sensor. Since the sensors were cheap ($5 for 5!) and I was going to have a spare ESP8266 lying around after my light sensor project, I figured what the heck, might as well pick some up and give it a go.

Wiring it up was again very straightforward. Ground to ground, VCC to a 3.3v pin, and D0 to one of the pins labelled D0-8. When you’re setting up ESPHome you can either convert the D pin to GPIO (refer to the above image for the pin layout on the NodeMCU) or just put in D whatever (ie D1, D2, D7, etc).

Here’s the code (based on the above GitHub):

binary_sensor:
 - platform: gpio
   pin: GPIO16
   name: "Dryer Vibration"
   device_class: vibration
   filters:
   - delayed_on: 15ms
   - delayed_off: 10s

Hit save, upload, and wait for the new image to be flashed to the ESP8266 and you should be set to go. Adding to HomeAssistant is similarly easy. But here’s where I started running into problems.

First I wanted to have a sensor that had a state of “Running” or “Not Running.” Simple, create a template sensor and can even customize the icon depending on state. Great. However I started noticing that the vibration sensor was sometimes not detecting vibration for periods of time or fluctuating back/forth. Also when the dryer door was opened/closed it would detect vibration. Not ideal.

I played with a number of solutions to solve this problem. First, placement of the sensor is crucial. Turn your dryer on and try to find a spot where the vibration seems higher and place the sensor there. Next, you can adjust the sensitivity of the sensor using a screwdriver and turning the little screw. For me I found that I had to turn up the sensitivity pretty high to get it to work.

That solves some of the problems, but I still had issues with the door opening/closing. My solution was to adjust my template sensor to require the binary vibration sensor to have last changed more than 30 seconds ago before the template sensor updates. So if the vibration sensor is “on” for 30 seconds it would change, or “off” for 30 seconds it would change. Any changes inside 30 seconds (like the door opening/closing) wouldn’t cause the template sensor to adjust. I also spent a LOT of time adjusting the filter values in ESP Home before settling on what I did (15ms on, 10s off). You’ll just need to play around with the different options to find what works for you.

Here’s my working template sensor for HomeAssistant:

      dryer_running:
        friendly_name: "Dryer Running"
        entity_id: sensor.time
        value_template: >
          {% if ((as_timestamp(now()) - as_timestamp(states.binary_sensor.dryer_vibration.last_changed)) > 30 | int ) and states('binary_sensor.dryer_vibration') == 'on'  %}
            Running
          {% else %}
            Not Running
          {% endif %}
        icon_template: >
          {% if states('binary_sensor.dryer_vibration') == 'on' %}
            mdi:tumble-dryer
          {% else %}
            mdi:tumble-dryer-off
          {% endif %}

(Almost) finished product in HASS. Likely will remove everything but the Status sensors eventually.

Now when the washer or dryer go from “Running” to “Not Running” the spouse and I get pings on our phone. Handy.

(I did washer sensing using a Tuya-converted power monitoring smart plug and the status template sensor is based on the watts being used by the washer).

Ease of project: 6.5/10

Again, very easy to wire. But the difficulty in getting it to work perfectly makes this slightly more complicated. And it’s still not perfect (there are a few false positive alerts that the washer/dryer is done), but given that the price was super cheap ($7 for the washer plug, $4 for the ESP8266 and $5 for the vibration sensors) it was worth it.

Bed Sensors

Following this guide I built a half decent bed sensor. This was my first experience soldering anything and it was surprisingly easy. The guide is pretty good at walking through what you need (note the need for an ESP32 and NOT an ESP8266. I missed that fact at first and had to wait for the ESP32 to be delivered). Making the sensor plates was pretty easy and the wiring is also pretty straightfoward once you get the soldering done. I would suggest writing down which plate you’re wiring to which pin so you aren’t like me and trying to figure out which plate is which after you installed everything (oops).

Sigh. Jumble of wires.

I’ve only had this up for a night but…it seems to work well. You’ll need to tweak the threshold setting to get it working but so far it’s pretty accurate. I’ll update this post if I find any other issues.

I’m really excited to have this one set up because I got tired of the spouse and I sitting in bed reading or watching TV and the lights turning off on us.

Ease of project: 7.75/10

Because it requires soldering I had to knock a bit off. It also involved a bit of effort to make the panels and also some tweaking to figure out the optimal settings. I also have an issue with the idea that this is a “$5 smart bed.” It is if you happen to have all the parts already lying around. But if you don’t, it’s more like $40/50. Still not that bad (and I have a spare ESP32 for another project, likely outdoor pathway lighting) but definitely not $5.

Conclusion

So that’s a few things you can use ESP8266 for. Honestly the options seem pretty much limitless. My next project is going to be LEDs for my office and outside pathway lights, but I’ve also got my eyes on this project to monitor salt levels in my water softner.

Alarm Control via HomeAssistant using Tasker, Join, and SleepAsAndroid

Wed, 27 May 2020 03:06:16 +0000

Introduction

There’s a million different ways of handling alarms to make sure you get up in the morning. Some people turn on the radio. Others need the lights to turn on bright. Still others need literal klaxons blaring in their ears. For me none of those options will work because usually I’m getting up early before my spouse to go for a run. So anything loud won’t work. A pretty good solution for me is using SleepAsAndroid tied to my Garmin watch so when my alarm goes off it first vibrates my watch. If I don’t turn the alarm off in ~2 minutes for some reason then it starts up with the progressively increasing in volume alarm sound.

Since I run, there are days when I don’t want to run such as when it’s too cold or pouring rain. I try to look at the weather forecast but sometimes the rain is scattered or it turns out not to be as cold as forecasted. So I wanted a way to automatically turn my early alarm off and turn on a later alarm, allowing me to sleep in. I also just wanted a way to adjust my alarm without having to pull out my phone, and with the recent COVID-19 outbreak adjust my alarm on the fly depending on my work schedule has proven useful.

There are 3 basic parts to this: 1) the ability to toggle an individual days alarm on/off, as well as the next day’s alarm, and then turn on a standard “sleep in” alarm, 2) the ability to turn off the next day’s alarm and set a custom alarm time, 3) the ability to turn on “vacation mode” and disable all the alarms and turn on an extra-late sleep in alarm (just in case!).

The below solution works for me and could probably be adjusted to work for you fairly easily. So far it’s been fairly bulletproof with some minor tweaks, but I would strongly suggest you test it carefully before you become 100% reliant on it.

Things we need (other than HomeAssistant and NodeRed):

Join - similar to apps like Pushbullet, this one is made by the same dev as Tasker. As a result, it has a unique feature we’ll need which is the ability to call Tasker activities remotely.
Tasker - a super handy app for any Android user. If you don’t have it, it’s worth the few dollars as it can pretty much do….anything. And the dev is always adding new features. This is Android only; sorry iOS users.
**SleepAsAndroid ** (any alarm app should work, so long as it supports some way of being controlled by Tasker, such as via intents)

HomeAssistant Setup

HomeAssistant is relatively simple and requires a number of input_booleans, input_numbers, and a template sensor (optional). Below is the code you can copy in your configuration.yaml.

input_number:
  alarmhour_custom:
    name: Hour
    icon: mdi:timer
    min: 1
    max: 12
    step: 1
  alarmminutes_custom:
    name: Minutes
    icon: mdi:timer
    min: 00
    max: 55
    step: 5
  alarm_custom:
    name: Custom
  alarm_disable_next:
    name: Skip Next Alarm

input_boolean:
  alarm_mon:
    name: Monday
  alarm_tue:
    name: Tuesday
  alarm_wed:
    name: Wednesday
  alarm_thu:
    name: Thursday
  alarm_fri:
    name: Friday
  alarm_sat:
    name: Saturday
  alarm_sun:
    name: Sunday
  alarm_vacation:
    name: Vacation
  alarm_custom:
    name: Custom
  alarm_disable_next:
    name: Skip Next Alarm
  work_home:
  	name: Pandemic Mode # aka Work from Home
    
sensor:
  - platform: template
    sensors:
      alarm_time_custom:
        friendly_name: 'Time'
        value_template: '{{ "%0.02d:%0.02d" | format(states("input_number.alarmhour_custom") | int, states("input_number.alarmminutes_custom") | int) }}'

The sensor is optional because I only use it to visually display my alarm time in Lovelace (see below). I use a custom card called Multiple Entity Row and I think it’s nice but not necessary for the core functionality we’ll discuss. Similarly optional, the “Pandemic Mode” is my COVID-19 addition that automatically turns off my early alarm at 8pm the day before. Eventually it will be converted to a “work from home” mode.

Here’s my Lovelace code, if you want to copy it as well:

  - badges: []
    cards:
      - entities:
          - input_boolean.alarm_mon
          - input_boolean.alarm_tue
          - input_boolean.alarm_wed
          - input_boolean.alarm_thu
          - input_boolean.alarm_fri
          - input_boolean.alarm_sat
          - input_boolean.alarm_sun
        show_header_toggle: false
        state_color: true
        title: Normal Alarms
        type: entities
      - entities:
          - entities:
              - sensor.alarm_time_custom
            entity: input_boolean.alarm_custom
            icon: 'mdi:alarm'
            name: Enable Custom Alarm
            toggle: true
            type: 'custom:multiple-entity-row'
          - entity: input_number.alarmhour_custom
          - entity: input_number.alarmminutes_custom
        show_header_toggle: false
        state_color: true
        title: Custom Alarm
        type: entities
      - entities:
          - input_boolean.alarm_disable_next
          - input_boolean.alarm_vacation
          - input_boolean.work_home
        show_header_toggle: false
        state_color: true
        title: Special Alarms
        type: entities
    path: alarm
    title: Alarm

That’s pretty much it for HomeAssistant, except for one more piece, which is Join.

Join

As mentioned above, Join is a Pushbullet-like service that is very useful for this situation mainly for the Tasker calls. Since this post is already going to be incredibly long I’m not going to go through all the details of setting up Join on your phone. I would suggest checking out the dev’s website here as I think it’s fairly self explanatory.

Once it’s set up on your phone, we’ll need to get things working with HomeAssistant. Check out the integration page here. Again, I think the docs are fairly self-explanatory here. The key things you need to find are your Join API key and device id. I would suggest just using a single device_id instead of the device_ids or device_names options. Here’s the relevant portion of my configuration.yaml:

joaoapps_join:
  - api_key: YOUR_JOIN_API_KEY
    device_id: YOUR_DEVICE_ID
    name: YOUR_DEVICE_NAME

Once you reboot HomeAssistant, you should have a bunch of new services. The key one we’ll need is joaoapps_join.YOUR_DEVICE_NAME_send_tasker. More on this later.

Tasker

If you haven’t used Tasker before, it’s really an amazing Android app that can do all kinds of things. Tasker has a bit of a learning curve, and setting it up is again outside the scope of this post. Just check out the subreddit or one of these articles for help.

Instead of copying the code into this post, see this gist. You shouldn’t need to make any changes to the code as it’s very generic, so long as you’re using SleepAsAndroid. If you’re using a different alarm app you’ll need to change the tasks to match your alarm app. You should still be able to use the trigger from Join without much change. Here’s what it should like when copied in.

Now what does this all do? Let’s just take the Normal Alarm Toggle profile as an example. When Join receives the Tasker Command “Alarm” along with the variables of alarmday and alarmtoggle (which we’ll get to/will come from NodeRed) it will then use a shell command to toggle the alarm on a specific day on/off. See the SleepAsAndroid API documentation for how I figured this part out. Don’t worry if you don’t understand any of that, again you shouldn’t have to do anything other than import the xml file. How to import? Open up Tasker, long press on the bottom edge and then click “Import Project.” Find where you saved the xml file and import it. Here’s a walk through if that doesn’t make sense.

SleepAsAndroid

For SleepAsAndroid you’ll need 2 alarms for every day: “mon” and another called “monsleep,” “tue” and “tuesleep” (note the name is important to work with the Tasker code above so follow this naming convention). You’ll also want to create an alarm called “Vacation.” Set the alarm times to whatever you want, that’s entirely up to you! Again not going into the finer details of setting up SleepAsAndroid here.

Note that in this example, my Wednesday alarm is turned off, but ‘wedsleep’ is turned on.

You also may need to enable Tasker in the SleepAsAndroid app. Go to Settings->Services and look for Tasker. I can’t remember if this is a requirement (I don’t think it is) so you can try without it first and then enable later if things aren’t working for you.

NodeRed

Now finally the fun part! This one could probably be simplified a bit on the NodeRed end but it works for me so it’s one of those “if it ain’t broke don’t fix it” deals. Basically what it does is whenever an alarm input_boolean changes, it runs through a bunch of checks and changes the data/payloads using some templates to get the data correct and then sends the Tasker command via Join. You may wonder why there are some rate limiting nodes: after some testing I discovered that Join/Tasker can’t handle a flood of requests all at once (most notably when turning vacation mode on/off) so some alarms weren’t being turned on/off correctly. Putting in the rate limits fixed those problems. So this isn’t instantaneous but it shouldn’t impact anything.

You will need to modify a few of the nodes to call the correct Join service. Everything else should be plug-n-play. The part in the bottom left corner is the reset that restores the normal alarm at a preset time of 11am. I picked this time kind of arbitrarily. My thought process was that if I used an earlier time I might inadvertently reset my alarm so it triggers again (ie I wake up earlier than normal I don’t want my alarm to go off twice!).

The NodeRed code is at the bottom of the post.

Conclusion

Phew. It may seem complicated (and let me tell you it was a pain to figure out all the bits and pieces, most notably the Tasker/Join bits), but you really should be able to use the code without much modification. The end result is a simple to control morning alarm system that you can use various automations (such as based on special calendar events/holidays, or other triggers) to easily modify your alarm schedule.

Hope you enjoyed. Feel free to message me over on Reddit (/u/FuzzyMistborn) if you have questions or run into issues!

[{"id":"7f30fabd.1fe11c","type":"comment","z":"b06c555f.69395","name":"Alarm","info":"","x":70,"y":740,"wires":[]},{"id":"5001a29f.98b48c","type":"server-state-changed","z":"b06c555f.69395","name":"Alarm Change","server":"63517380.eb951c","version":1,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"input_boolean.alarm_","entityidfiltertype":"substring","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":true,"x":90,"y":840,"wires":[["5163d1d5.eb669"]]},{"id":"37479bd2.d12044","type":"api-call-service","z":"b06c555f.69395","name":"Vacation Off","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_on","entityId":"input_boolean.alarm_mon, input_boolean.alarm_tue, input_boolean.alarm_wed, input_boolean.alarm_thu, input_boolean.alarm_fri, input_boolean.alarm_sat, input_boolean.alarm_sun","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":510,"y":920,"wires":[["4628cf9a.9406d8"]]},{"id":"790891a8.3b6418","type":"switch","z":"b06c555f.69395","name":"Switch","property":"data.entity_id","propertyType":"msg","rules":[{"t":"eq","v":"input_boolean.alarm_mon","vt":"str"},{"t":"eq","v":"input_boolean.alarm_tue","vt":"str"},{"t":"eq","v":"input_boolean.alarm_wed","vt":"str"},{"t":"eq","v":"input_boolean.alarm_thu","vt":"str"},{"t":"eq","v":"input_boolean.alarm_fri","vt":"str"},{"t":"eq","v":"input_boolean.alarm_sat","vt":"str"},{"t":"eq","v":"input_boolean.alarm_sun","vt":"str"},{"t":"eq","v":"input_boolean.alarm_disable_next","vt":"str"},{"t":"eq","v":"input_boolean.alarm_vacation","vt":"str"},{"t":"eq","v":"input_boolean.alarm_custom","vt":"str"}],"checkall":"true","repair":false,"outputs":10,"x":290,"y":840,"wires":[["9e45cc70.640eb8"],["9e45cc70.640eb8"],["9e45cc70.640eb8"],["9e45cc70.640eb8"],["9e45cc70.640eb8"],["9e45cc70.640eb8"],["9e45cc70.640eb8"],["2dbfb2f7.fcacae"],["34d9e1be.312416"],["5cea82b2.b77f1c"]]},{"id":"34d9e1be.312416","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"off","vt":"str"},{"t":"eq","v":"on","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":395,"y":940,"wires":[["37479bd2.d12044"],["627c2fbc.2766f8"]],"l":false},{"id":"627c2fbc.2766f8","type":"api-call-service","z":"b06c555f.69395","name":"Vacation On","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_off","entityId":"input_boolean.alarm_mon, input_boolean.alarm_tue, input_boolean.alarm_wed, input_boolean.alarm_thu, input_boolean.alarm_fri, input_boolean.alarm_sat, input_boolean.alarm_sun","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":510,"y":960,"wires":[["4628cf9a.9406d8"]]},{"id":"90ffc709.75d6b8","type":"api-render-template","z":"b06c555f.69395","name":"Tomorrow","server":"63517380.eb951c","template":"{{(as_timestamp(now())+ (86400)) | timestamp_custom(\"%a\",true) | lower}}","resultsLocation":"result","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":800,"y":840,"wires":[["1a91437a.275f6d"]]},{"id":"541013ae.45db2c","type":"api-call-service","z":"b06c555f.69395","name":"Enable Next","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_on","entityId":"input_boolean.alarm_{{result}}","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":1030,"y":820,"wires":[[]]},{"id":"57999dbf.ed9304","type":"api-call-service","z":"b06c555f.69395","name":"Disable Next","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_off","entityId":"input_boolean.alarm_{{result}}","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":1030,"y":860,"wires":[[]]},{"id":"1a91437a.275f6d","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"off","vt":"str"},{"t":"eq","v":"on","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":915,"y":840,"wires":[["541013ae.45db2c"],["57999dbf.ed9304"]],"l":false},{"id":"a3c555f8.a571d","type":"api-call-service","z":"b06c555f.69395","name":"Vacation Alarm","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"joaoapps_join","service":"send_tasker","entityId":"","data":"{ \"command\": \"Alarm Vacation {{payload}}\" }","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":820,"y":940,"wires":[[]]},{"id":"4628cf9a.9406d8","type":"delay","z":"b06c555f.69395","name":"","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":635,"y":940,"wires":[["a3c555f8.a571d"]],"l":false},{"id":"9e45cc70.640eb8","type":"change","z":"b06c555f.69395","name":"Change","rules":[{"t":"change","p":"data.entity_id","pt":"msg","from":"input_boolean.alarm_","fromt":"re","to":"","tot":"str"},{"t":"set","p":"day","pt":"msg","to":"data.entity_id","tot":"msg"},{"t":"change","p":"payload","pt":"msg","from":"off","fromt":"str","to":"false","tot":"str"},{"t":"change","p":"payload","pt":"msg","from":"on","fromt":"str","to":"true","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":440,"y":780,"wires":[["1939c7cc.46762"]]},{"id":"500b1ff.cdf906","type":"api-current-state","z":"b06c555f.69395","name":"Vacation","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.alarm_vacation","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":true,"x":620,"y":840,"wires":[["90ffc709.75d6b8"],["1f231516.02baab"]]},{"id":"1f231516.02baab","type":"api-call-service","z":"b06c555f.69395","name":"Turn off Skip","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_off","entityId":"input_boolean.alarm_disable_next","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":810,"y":900,"wires":[[]]},{"id":"1939c7cc.46762","type":"api-call-service","z":"b06c555f.69395","name":"Alarm Change","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"joaoapps_join","service":"send_tasker","entityId":"","data":"{ \"command\": \"Alarm setvar=:={{day}}=:={{payload}}\" }","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":600,"y":780,"wires":[["3bb7480c.5bbf48"]]},{"id":"663722e9.652aec","type":"api-call-service","z":"b06c555f.69395","name":"Sleep Enable","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"joaoapps_join","service":"send_tasker","entityId":"","data":"{ \"command\": \"Alarm setvar=:={{day}}sleep=:=true\" }","dataType":"json","mergecontext":"","output_location":"final","output_location_type":"msg","mustacheAltTags":false,"x":1050,"y":720,"wires":[[]]},{"id":"71ae75ef.9db39c","type":"api-call-service","z":"b06c555f.69395","name":"Sleep Disable","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"joaoapps_join","service":"send_tasker","entityId":"","data":"{ \"command\": \"Alarm setvar=:={{day}}sleep=:=false\" }","dataType":"json","mergecontext":"","output_location":"final","output_location_type":"msg","mustacheAltTags":false,"x":1060,"y":760,"wires":[[]]},{"id":"9dd3ef67.fcb088","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"false","vt":"str"},{"t":"eq","v":"true","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":935,"y":740,"wires":[["663722e9.652aec"],["71ae75ef.9db39c"]],"l":false},{"id":"f461cc00.ec2b38","type":"api-call-service","z":"b06c555f.69395","name":"Custom Off","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_off","entityId":"input_boolean.alarm_custom, input_boolean.alarm_disable_next","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":390,"y":1020,"wires":[[]]},{"id":"b8c812bf.f6607","type":"api-render-template","z":"b06c555f.69395","name":"Today","server":"63517380.eb951c","template":"{{ (as_timestamp(now())) | timestamp_custom(\"%a\",true) | lower }}","resultsLocation":"result","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":90,"y":1080,"wires":[["8b7339e2.38cc5"]]},{"id":"c4eb19ca.8e677","type":"api-call-service","z":"b06c555f.69395","name":"Reenable Alarm","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_on","entityId":"input_boolean.alarm_{{result}}","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":380,"y":1080,"wires":[[]]},{"id":"2dbfb2f7.fcacae","type":"api-current-state","z":"b06c555f.69395","name":"Custom","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.alarm_custom","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":true,"x":440,"y":840,"wires":[["500b1ff.cdf906"],[]]},{"id":"3bb7480c.5bbf48","type":"api-render-template","z":"b06c555f.69395","name":"Alarm Check","server":"63517380.eb951c","template":"{{ states('input_boolean.alarm_custom') == \"off\" and states('input_boolean.alarm_vacation') == \"off\" }}","resultsLocation":"template","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":810,"y":780,"wires":[["c354d9be.949cb8"]]},{"id":"8b7339e2.38cc5","type":"api-current-state","z":"b06c555f.69395","name":"Vacation","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.alarm_vacation","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":true,"x":220,"y":1080,"wires":[["c4eb19ca.8e677"],[]]},{"id":"66de9a0b.b4bf5c","type":"api-current-state","z":"b06c555f.69395","name":"Vacation","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.alarm_vacation","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":true,"x":800,"y":1120,"wires":[[],["a76aa273.8a0f8"]]},{"id":"a76aa273.8a0f8","type":"change","z":"b06c555f.69395","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"on","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":895,"y":1120,"wires":[["73f26c6d.a40db4"]],"l":false},{"id":"4a0a3830.0b41a8","type":"api-render-template","z":"b06c555f.69395","name":"Hour","server":"63517380.eb951c","template":"{{ states.input_number.alarmhour_custom.state | int }}","resultsLocation":"hour","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":790,"y":1020,"wires":[["e515bb17.f964c"]]},{"id":"e515bb17.f964c","type":"api-render-template","z":"b06c555f.69395","name":"Minute","server":"63517380.eb951c","template":"          {% if states.input_number.alarmminutes_custom.state | int == 0 %}\n            00\n          {% elif states.input_number.alarmminutes_custom.state | int == 5 %}\n            05\n          {% else %}\n            {{ states.input_number.alarmminutes_custom.state | int }}\n          {% endif %}","resultsLocation":"minute","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":910,"y":1020,"wires":[["c5042d57.e7d9f"]]},{"id":"c5042d57.e7d9f","type":"api-call-service","z":"b06c555f.69395","name":"Custom Alarm","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"joaoapps_join","service":"send_tasker","entityId":"","data":"{ \"command\": \"Custom setvar=:={{hour}}=:={{minute}}\" }","dataType":"json","mergecontext":"","output_location":"final","output_location_type":"msg","mustacheAltTags":false,"x":1060,"y":1020,"wires":[["500b1ff.cdf906"]]},{"id":"5cea82b2.b77f1c","type":"switch","z":"b06c555f.69395","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"on","vt":"str"},{"t":"eq","v":"off","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":650,"y":1020,"wires":[["4a0a3830.0b41a8","421f472e.7f727"],["66de9a0b.b4bf5c","5ca46a08.9a68f4"]]},{"id":"421f472e.7f727","type":"api-current-state","z":"b06c555f.69395","name":"Vacation","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.alarm_vacation","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":true,"x":800,"y":1080,"wires":[[],["8d0081eb.831bb8"]]},{"id":"8d0081eb.831bb8","type":"change","z":"b06c555f.69395","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"off","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":895,"y":1080,"wires":[["73f26c6d.a40db4"]],"l":false},{"id":"73f26c6d.a40db4","type":"api-call-service","z":"b06c555f.69395","name":"Vacation Alarm","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"joaoapps_join","service":"send_tasker","entityId":"","data":"{ \"command\": \"Alarm Vacation Custom {{payload}}\" }","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":1020,"y":1100,"wires":[[]]},{"id":"dc1752fe.9078f","type":"schedex","z":"b06c555f.69395","name":"Restore Sleep In","suspended":false,"lat":"","lon":"","ontime":"11:00","ontopic":"","onpayload":"","onoffset":0,"onrandomoffset":0,"offtime":"","offtopic":"","offpayload":"","offoffset":0,"offrandomoffset":0,"mon":true,"tue":true,"wed":true,"thu":true,"fri":true,"sat":true,"sun":true,"x":110,"y":1020,"wires":[["b8c812bf.f6607","f461cc00.ec2b38"]]},{"id":"2f8f5337.51397c","type":"delay","z":"b06c555f.69395","name":"","pauseType":"delay","timeout":"15","timeoutUnits":"seconds","rate":"1","nbRateUnits":"30","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":875,"y":740,"wires":[["9dd3ef67.fcb088"]],"l":false},{"id":"5163d1d5.eb669","type":"delay","z":"b06c555f.69395","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"15","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":195,"y":840,"wires":[["790891a8.3b6418"]],"l":false},{"id":"c354d9be.949cb8","type":"switch","z":"b06c555f.69395","name":"","property":"template","propertyType":"msg","rules":[{"t":"eq","v":"True","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":915,"y":780,"wires":[["2f8f5337.51397c"]],"l":false},{"id":"5ca46a08.9a68f4","type":"api-current-state","z":"b06c555f.69395","name":"Vacation","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"off","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.alarm_vacation","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":true,"x":800,"y":1160,"wires":[["f8830245.577a38"],[]]},{"id":"f8830245.577a38","type":"api-render-template","z":"b06c555f.69395","name":"Tomorrow","server":"63517380.eb951c","template":"{{(as_timestamp(now())+ (86400)) | timestamp_custom(\"%a\",true) | lower}}","resultsLocation":"result","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":940,"y":1160,"wires":[["9fdadc80.b6388"]]},{"id":"9fdadc80.b6388","type":"api-call-service","z":"b06c555f.69395","name":"Enable Next","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"input_boolean","service":"turn_on","entityId":"input_boolean.alarm_{{result}}","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":1090,"y":1160,"wires":[[]]},{"id":"63517380.eb951c","type":"server","z":"","name":"Home Assistant","legacy":false,"rejectUnauthorizedCerts":false,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

Automating Vacation Mode

Mon, 18 May 2020 20:28:46 +0000

Last week I wrote up about how to set up vacation mode for your lights easily. This week we’re going to complete the loop a bit and automate turning on vacation mode. Note this setup requires an Ecobee thermostat. It’s a popular thermostat so this may be of some use to folks. I like the Ecobee vacation mode because it allows me to set up a permanent hold whenever I’m not at home for an extended period of time, which usually coincides with me being on vacation/away from home. Thanks to HomeAssistant you can set up a vacation on your Ecobee from a Google Calendar or other supported calendar (I used CalDAV).

Now you can set up a vacation from either the app or on the Ecobee website. So you may be asking “why do this?” The answer is I’m lazy and I hate having to open another app/webpage when I already put vacations in my calendar. So why not automate it at that point?

Calendar

There are two main calendar options: CalDAV and Google Calendar. The HASS documentation is pretty decent for these so I’m not going to spend a lot of time on how to set them up. For our purposes it doesn’t matter which one you use, just use whichever one you use/are most comfortable with.

Once you get your calendars imported, find the entity_id for your vacation calendar. I find it useful to have a separate calendar for vacation but you could also use another calendar and just use a search/regex to pull out your vacation entities. Either way, look for a calendar. entry in your states page of HASS and use that for the below automations.

Automation

Since I use NodeRed the below is how I got things to work. I’ll share an automation for HomeAssistant that should work but I don’t make any promises. Here’s a visual of what the flow looks like (code will be towards the end of the post).

The first node is simply a trigger node to start the flow when the calendar in HomeAssistant goes from “off” to “on” (ie the calendar event is starting). You will need to change the entity_id to match your calendar’s name.

The next 5 flows use templates to set the various variables we will need later when calling the ecobee.create_vacation service call in HomeAssistant. You will need to adjust the calendar name in each node to match your own calendar name. For the Function node, you will need to adjust the entity_id of the thermostat to yours and also adjust the temperature presets (can be in Celsius or Fahrenheit). You’ll need to do the same in the Create Vacation service node.

That’s it. Now whenever the calendar reaches an event on your vacation calendar, the Ecobee vacation mode will automatically be set for you. I then use proximity to trigger a flow/automation to turn on my vacation input_boolean when I get more than 20 miles from home and the Ecobee is in vacation mode. And then same works in reverse (when I go under 20 miles and the input_boolean is on, it turns off the input_boolean).

NodeRed Code Export

[{"id":"455725fe.32e6a4","type":"comment","z":"f89809b5.67a0a8","name":"Vacation Mode","info":"","x":860,"y":760,"wires":[]},{"id":"ceef80b3.bd2ec","type":"trigger-state","z":"f89809b5.67a0a8","name":"Vacay Start","server":"63517380.eb951c","exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityid":"calendar.vacation","entityidfiltertype":"exact","debugenabled":false,"constraints":[{"id":"yi1ia7zhq3q","targetType":"this_entity","targetValue":"","propertyType":"current_state","propertyValue":"new_state.state","comparatorType":"is","comparatorValueDatatype":"str","comparatorValue":"on"},{"id":"ywqkg3b9vvn","targetType":"this_entity","targetValue":"","propertyType":"previous_state","propertyValue":"old_state.state","comparatorType":"is","comparatorValueDatatype":"str","comparatorValue":"off"}],"constraintsmustmatch":"all","outputs":2,"customoutputs":[],"outputinitially":false,"state_type":"str","x":850,"y":820,"wires":[["9409550f.7ac018"],[]]},{"id":"57cb7aa3.2f8224","type":"api-call-service","z":"f89809b5.67a0a8","name":"Create Vacation","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"ecobee","service":"create_vacation","entityId":"","data":"{\"entity_id\":\"climate.ecobee\",\"vacation_name\":\"{{payload.vacation_name}}\",\"cool_temp\":\"78\",\"heat_temp\":\"66\",\"start_date\":\"{{payload.start_date}}\",\"start_time\":\"{{payload.start_time}}\",\"end_date\":\"{{payload.end_date}}\",\"end_time\":\"{{payload.end_time}}\"}","dataType":"json","mergecontext":"","output_location":"payload","output_location_type":"msg","mustacheAltTags":false,"x":1500,"y":780,"wires":[[]]},{"id":"9409550f.7ac018","type":"api-render-template","z":"f89809b5.67a0a8","name":"Vacay Name","server":"63517380.eb951c","template":"{{ state_attr('calendar.vacation', 'message') }}","resultsLocation":"vacay_name","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":1110,"y":740,"wires":[["2a3c6979.d3f7e6"]]},{"id":"2a3c6979.d3f7e6","type":"api-render-template","z":"f89809b5.67a0a8","name":"Start Date","server":"63517380.eb951c","template":"{{ as_timestamp(state_attr('calendar.vacation', 'start_time')) | timestamp_custom(\"%Y-%m-%d\") }}","resultsLocation":"start_date","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":1100,"y":780,"wires":[["f1e211f.564267"]]},{"id":"f1e211f.564267","type":"api-render-template","z":"f89809b5.67a0a8","name":"Start Time","server":"63517380.eb951c","template":"{{ as_timestamp(state_attr('calendar.vacation', 'start_time')) | timestamp_custom(\"%H:%M:%S\") }}","resultsLocation":"start_time","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":1110,"y":820,"wires":[["227d98a3.b79e78"]]},{"id":"227d98a3.b79e78","type":"api-render-template","z":"f89809b5.67a0a8","name":"End Date","server":"63517380.eb951c","template":"{{ as_timestamp(state_attr('calendar.vacation', 'end_time')) | timestamp_custom(\"%Y-%m-%d\") }}","resultsLocation":"end_date","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":1300,"y":740,"wires":[["c292db2d.1b8f38"]]},{"id":"c292db2d.1b8f38","type":"api-render-template","z":"f89809b5.67a0a8","name":"End Time","server":"63517380.eb951c","template":"{{ as_timestamp(state_attr('calendar.vacation', 'end_time')) | timestamp_custom(\"%H:%M:%S\") }}","resultsLocation":"end_time","resultsLocationType":"msg","templateLocation":"","templateLocationType":"none","x":1300,"y":780,"wires":[["cf86984c.4d0d18"]]},{"id":"cf86984c.4d0d18","type":"function","z":"f89809b5.67a0a8","name":"Set Payload","func":"newmsg = {}\n\nnewmsg.payload = { \"entity_id\": \"climate.ecobee\", \"vacation_name\": (msg.vacay_name), \"cool_temp\":\"78\", \"heat_temp\":\"62\", \"start_date\": (msg.start_date), \"start_time\": (msg.start_time), \"end_date\": (msg.end_date), \"end_time\": (msg.end_time) }\n\nreturn newmsg;\n","outputs":1,"noerr":0,"x":1310,"y":820,"wires":[["57cb7aa3.2f8224"]]},{"id":"63517380.eb951c","type":"server","z":"","name":"Home Assistant","legacy":false,"rejectUnauthorizedCerts":false,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

HomeAssistant Automation

- alias: "Ecobee Create Vacation"
  trigger:
    platform: state
    entity_id: calendar.YOUR_CALENDAR
    from: "off"
    to: "on"
  action:
    service: ecobee.create_vacation
    data_template:
      entity_id: climate.YOUR_THERMOSTAT
      vacation_name: "{{ state_attr('calendar.YOUR_CALENDAR', 'message') }}"
      cool_temp: 78
      heat_temp: 62
      start_date: "{{ as_timestamp(state_attr('calendar.YOUR_CALENDAR', 'start_time')) | timestamp_custom('%Y-%m-%d') }}"
      start_time: "{{ as_timestamp(state_attr('calendar.YOUR_CALENDAR', 'start_time')) | timestamp_custom('%H:%M:%S') }}"
      end_date: "{{ as_timestamp(state_attr('calendar.YOUR_CALENDAR', 'end_time')) | timestamp_custom('%Y-%m-%d') }}"
      end_time: "{{ as_timestamp(state_attr('calendar.YOUR_CALENDAR', 'end_time')) | timestamp_custom('%H:%M:%S') }}"

Change “YOUR_THEROMSTAT” and “YOUR_CALENDAR” to match accordingly

Easy Smart Vacation Mode Lighting with HomeAssistant

Sun, 10 May 2020 18:42:49 +0000

Lighting definitely helps to deter burglaries when you aren’t home. I’m old enough to remember having to pull out those plug in timers to turn a lamp on/off on a timer whenever my family would go on vacation. But the issue there is that 1) it was a fixed time and 2) only a few lights in the house were able to do this. The other option was to leave lights on all the time, but that’s both wasteful and not very effective against convincing burglars that you’re actually home.

There’s obviously a lot of ways to do this. The most common is to randomize the lights turning on/off between set times (say sunset and 10pm). I initially went down this path using Occusim but I hit 2 problems. First, it requires AppDaemon which is a great tool for doing things in Python. However, OccuSim was the only thing I was using AppDaemon for and I generally don’t like one-off solutions like this because it’s yet another thing to update and more importantly it required a lot of tweaking to get “right.” I had to think about where I was at different times of day which isn’t always the same. So while it allowed me to randomize lights on a schedule or at random times overall it wasn’t a great solution.

Replay Last Week’s Light Schedule

By far the best way I’ve found to do this is to “replay” what the lights did the previous week. So if on Saturday the lights in my bedroom turned on at 6:51am and off at 7:23am, then with my vacation mode setup the lights in the bedroom would do the exact same thing a week later. By mimicking what I did in a past week it will 1) reflect a real schedule, 2) saves me from having to pre-program based on weekday/weekend, take into account the sunset, etc, and 3) with it being a week apart it’s unlikely a criminal would realize that the lights are following any kind of pre-determined program. Hat tip to /u/atra-ignis on Reddit for the initial concept and helping me get this working. I did make some tweaks from there but credit for the idea goes to them.

Setup

The two tools we need for this are HomeAssistant and NodeRed (though you can do the automation bit in HomeAssistant too if that’s what you use). This presumes you have all your lights set up in HomeAssistant, obviously. Also, you’ll need to have your recorder/database set up to record for a least 1 week.

HomeAssistant

For this, we’re going to use the History Stats integration. This integration has some pretty cool use cases but here what we’ll use it for is to create a sensor that will represent with a 0 or a 1 whether a light is on or off. Here is an example:

sensor:
  - platform: history_stats
    name: "Replay Office"
    entity_id: light.office
    state: "on"
    type: count
    start: >
      {{ as_timestamp(now()) - (7*86400) }}
    duration: 00:00:30

Goes in the sensor portion of your configuration.yaml file

A lot of the above is self explanatory. Name the sensor whatever you want, though I would strongly suggest starting with some of kind of prefix like “Replay” or “Vacation” which you’ll see why when we get to NodeRed. With the history_stats sensor you can define 2 of start, end or duration. For our purposes we’re using start and duration. The start time is 7 days ago, so we create a template to determine that (there are 86400 seconds in a day). Duration is how used to measure. I keep mine to 30 seconds to get a fairly accurate picture of the changes.

Create a sensor for each light/switch/group you want to track. Restart HomeAssistant and you should see something like this when you click on the sensor in the States page:

1 represent ‘on’ and 0 represents ‘off’

You’ll also want some way of tracking whether you’re on vacation or not. For me I use an input_boolean called “Vacation Mode” (original I know) that I can manually toggle on/off, but actually is triggered by my Ecobee thermostat going into vacation mode. You can set up vacations manually in the Ecobee app or what I do is create the vacation based on a vacation calendar in Google and use that information to trigger the vacation mode in Ecobee.

NodeRed

The NodeRed setup is extremely simple. I’ll post the code below but here’s a brief explainer of what it does.

Vacation mode made simple

The first node is a state node that triggers when any of the replay sensor changes. Instead of having to create a node for every single sensor, if you followed the naming suggestion I made above and everything is sensor.replay_ then you can use a single node. Just select “substring” instead of “exact” in the node settings and don’t put anything for the state.

Select ‘substring’

The next node is a check to make sure that vacation mode is turned on. Then there’s a change node to change the “1” payload to “on” and the “0” payload to “off.” Finally there’s a switch node to route the changes to the right light/switch you want to control, depending on which sensor.replay_ is changing, with a corresponding call service node to turn the light on/off depending on the payload.

Using {{payload}} allows you to have one node for turn on and off

As promised, here’s the NodeRed code:

[{"id":"83bc26c9.2c2288","type":"server-state-changed","z":"9a7feaee.12c878","name":"Replay","server":"63517380.eb951c","version":1,"entityidfilter":"sensor.replay_","entityidfiltertype":"substring","outputinitially":false,"state_type":"str","haltifstate":"","halt_if_type":"str","halt_if_compare":"is","outputs":1,"output_only_on_state_change":true,"x":250,"y":4300,"wires":[["7081a406.114374"]]},{"id":"7081a406.114374","type":"api-current-state","z":"9a7feaee.12c878","name":"Vacation Mode?","server":"63517380.eb951c","version":1,"outputs":2,"halt_if":"on","halt_if_type":"str","halt_if_compare":"is","override_topic":false,"entity_id":"input_boolean.vacation_mode","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":400,"y":4300,"wires":[["89580e3b.3d991"],[]]},{"id":"89580e3b.3d991","type":"change","z":"9a7feaee.12c878","name":"State","rules":[{"t":"change","p":"payload","pt":"msg","from":"1","fromt":"num","to":"on","tot":"str"},{"t":"change","p":"payload","pt":"msg","from":"0","fromt":"num","to":"off","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":550,"y":4300,"wires":[["261a9672.986142"]]},{"id":"261a9672.986142","type":"switch","z":"9a7feaee.12c878","name":"Switch","property":"data.entity_id","propertyType":"msg","rules":[{"t":"eq","v":"sensor.replay_basement","vt":"str"},{"t":"eq","v":"sensor.replay_office","vt":"str"},{"t":"eq","v":"sensor.replay_kitchen","vt":"str"}],"checkall":"true","repair":false,"outputs":3,"x":670,"y":4300,"wires":[["bfe718.3fb748e8"],["3fdd00a1.a644f"],["b40d6311.da7848"]]},{"id":"df4ffbf5.3de44","type":"comment","z":"9a7feaee.12c878","name":"Vacation Mode - Replay","info":"","x":300,"y":4200,"wires":[]},{"id":"bfe718.3fb748e8","type":"api-call-service","z":"9a7feaee.12c878","name":"Basement","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"light","service":"turn_{{payload}}","entityId":"light.basement","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":820,"y":4260,"wires":[[]]},{"id":"3fdd00a1.a644f","type":"api-call-service","z":"9a7feaee.12c878","name":"Office","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"light","service":"turn_{{payload}}","entityId":"light.office","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":810,"y":4300,"wires":[[]]},{"id":"b40d6311.da7848","type":"api-call-service","z":"9a7feaee.12c878","name":"Kitchen","server":"63517380.eb951c","version":1,"debugenabled":false,"service_domain":"switch","service":"turn_{{payload}}","entityId":"switch.kitchen","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":820,"y":4340,"wires":[[]]},{"id":"63517380.eb951c","type":"server","z":"","name":"Home Assistant","legacy":false,"rejectUnauthorizedCerts":false,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

Conclusion

That’s it. Now whenever you turn on vacation mode in HomeAssistant the lights will simply repeat what they did the week before. Even if you’re on vacation for multiple weeks it won’t matter, it will just repeat what the lights did the previous week (which is the same as the previous week). While some very clever criminal might pick up on the fact the lights keep repeating exactly the same week in, week out that’s fairly unlikely.

Wink Subscription Service...AKA why the cloud sucks

Thu, 07 May 2020 18:50:17 +0000

Reasons why not to rely on a cloud service for running your smart home #23487834208: yesterday Wink announced a new subscription service. This subscription takes effect in just 7 days and will be required in order to “access your Wink devices from the app, with voice control or through the API, and [operate] your automations.” Or….you know, the entire point of a smart home hub. So if you don’t pay your device turns into a useless paperweight.

Nice new paperweights

I’m not going to pull out the pitchfork too much to go after Wink for this. I totally understand they need to make money and I think they’re trying to keep the lights on so they don’t have to completely fold. But that doesn’t justify giving folks only 7 days notice of such a significant change. They should have grandfathered existing users in for a month of free service before requiring payment. And to all the armchair attorneys out there talking about claims of criminal fraud or class action lawsuits, all I’ll say is good luck going after a company that’s for all intents and purposes bankrupt.

Sometimes you just can’t avoid the cloud and I totally get that. But for all those Wink users out there who are thinking of jumping over to SmartThings you should think hard on that one whether you want to jump into bed with another cloud provider that could do the same thing a year or two from now. Not saying that Samsung will but they could and as this Wink experience shows, there’s pretty much not a thing you can do about it.

SSH Security - Limited Commands

Sat, 02 May 2020 17:38:18 +0000

SSH (Secure Shell) is pretty much a necessity in any kind of smart home set up, particularly if you’re running Linux. It’s the main way that you connect to your computer (presuming its not your main desktop but some kind of server/Raspberry Pi/NUC/other computer). Because SSH is so crucial, it’s also a potential vector for hackers to gain access to your system so you really need to make sure it’s protected.

I don’t really want to spend a lot of time on the standard practices to secure SSH. There are plenty of articles out there that cover this. My personal favorite is from How-to Geek here. At a minimum, you need to disable root ssh login, turn off password authentication and only use SSH keys. Presuming you’ve done that, let’s take it a step further and let’s restrict what terminal commands an SSH key can perform.

You may ask why you’d want to do this. An easy example is the one I alluded to in my post about CATT. I run HomeAssistant in a Docker container and CATT runs on my server outside of a container. To run the CATT commands I therefore needed to SSH from the Docker container to the server. That obviously presents a security risk because one of the main benefits of Docker is that it sandboxes things a bit to limit security issues. So if there’s a vulnerability in one of your containers your entire server isn’t necessarily compromised (though it still could be, always keep your services up to date!). So I needed the ability to SSH to my server but I didn’t want HASS to have unrestricted access should the SSH key HASS uses somehow get compromised.

Turns out it’s fairly easy to limit what commands an SSH key can run. All you need to do is add the following bit of code to your authorized_keys file (in your .ssh folder) at the very begining of the line that contains the SSH key you’re trying to restrict.

command="ping 192.168.1.1" ssh-rsa AAB34...

Everything beyond ssh-rsa is purely an example. Keep whatever values are there.

Now, that key can only run ping 192.168.1.1 command when you try to use it. So for example if you’re logging into a Raspberry Pi via ssh and try ssh pi@192.168.1.2 ls instead of getting a list of files it will start pinging 192.168.1.1. Here’s the catch though. You can only list one command. So if you want the SSH key to be able to run multiple commands you need to do a bit more work.

The way around the single command limitation is to create a bash script that you put in the authorized_keys file. Here’s how to do it. Open up your text editor of choice (I like nano personally so I would do nano ~/.ssh/auth_commands.sh) and create a file in your .ssh directory like this:

#!/bin/sh
#
# You can have only one forced command in ~/.ssh/authorized_keys. Use this
# wrapper to allow several commands.

case "$SSH_ORIGINAL_COMMAND" in
    "ping 192.168.1.1")
        ping 192.168.1.1
        ;;
    "ping 192.168.1.3")
        ping 192.168.1.3
        ;;
    "ls /home/pi/docker")
        ls /home/pi/docker
        ;;
    *)
        echo "Access denied"
        exit 1
        ;;
esac

Obviously you’ll want to customize to whatever commands you want. Depending on the command you may need to specify the full path to the command (run which command_name to get the path to the command). Next you need to make the file executable, so chmod +x ~/.ssh/auth_commands.sh. Then you’ll need to modify your authorized_keys files to point to the auth_commands.sh file (ie command="~./ssh/auth_commands.sh").

Now when you run ssh pi@192.168.1.2 ls /home/pi/docker you’ll get the results of the command. But if you try ssh pi@192.168.1.2 ls /home/pi/homeassistant instead you’ll get “Access denied” in response.

Obviously most of the time when you create SSH keys you’re going to want to allow full access to the computer. But if you want to limit access for some reason, I’ve found the above to be the simpliest and easiest way to accomplish that goal.

HomeAssistant and CATT (Cast All the Things!)

Fri, 01 May 2020 01:28:25 +0000

Home Assistant Cast was announced about 9 months ago (August 2019) and it’s a very nifty concept if you have a Google/Nest Home Hub. Essentially it allows you to cast a Lovelace page to your Hub and you can use the touch screen to interact with the page, just like a tablet. There are a lot of possible use cases, from a security camera monitor to light/room controls and even media controls. One example I use is to pull up a Google Maps view of my commute that shows me traffic data, and another example is to pull up a radar image on my nightstand so I can easily see if it’s raining when I wake up and decide whether I should go back to bed or go for a run.

Unfortunately for me I could never get it to work consistently enough. A common scenario would be I either got just a blank screen or I would get a HomeAssistant screen with “Connected” but no Lovelace page or even an error that it couldn’t find the path, despite the fact that it had worked before and I had not changed anything on my end. This Github issue is representative of the many issues I had.

There’s also issues that you need your HASS installation to be remotely accessible (either through NabuCasa or your own webpage/dynamic DNS url) which isn’t an issue for many but for some it might be.

Then one day I stumbled across this thread in the HASS Community forum. There’s an alternative out there called, appropriately, “Cast All the Things” and it’s available on Github. Let’s go through the steps to get it installed and working with HASS.

Installation

Installation is easy enough. Open up a terminal on whatever device you use to run HASS (if you’re running HASS.IO/HomeAssistant you probably need to use a different machine. I don’t use Hass.io so can’t help with the install there check out this GitHub which should allow for CATT control via HASS. Please note I haven’t used so can’t speak to how well it works). The install command is:

pip3 install catt

You may need to make it “pip” depending on how you have Python/pip installed

Once installed, using CATT is fairly straightforward. Here’s a list of all the commands:

Commands:
  add           Add a video to the queue (YouTube only).
  cast          Send a video to a Chromecast for playing.
  cast_site     Cast any website to a Chromecast.
  clear         Clear the queue (YouTube only).
  del_alias     Delete the alias name of the selected device.
  del_default   Delete the default device.
  ffwd          Fastforward a video by TIME duration.
  info          Show complete information about the currently-playing video.
  pause         Pause a video.
  play          Resume a video after it has been paused.
  play_toggle   Toggle between playing and paused state.
  remove        Remove a video from the queue (YouTube only).
  restore       Return Chromecast to saved state.
  rewind        Rewind a video by TIME duration.
  save          Save the current state of the Chromecast for later use.
  scan          Scan the local network and show all Chromecasts and their IPs.
  seek          Seek the video to TIME position.
  set_alias     Set an alias name for the selected device.
  set_default   Set the selected device as default.
  skip          Skip to end of content.
  status        Show some information about the currently-playing video.
  stop          Stop playing.
  volume        Set the volume to LVL [0-100].
  volumedown    Turn down volume by a DELTA increment.
  volumeup      Turn up volume by a DELTA increment.
  write_config  Please use "set_default".

I would suggest starting with running “catt scan” to make sure you can find all your Chromecast devices. Then you can test it out by running:

catt -d "NAME_OF_DEVICE" cast_site https://www.google.com

“NAME_OF_DEVICE” is whatever the name of the device is from running “catt scan”

You should be able to see the Google webpage on your Hub. You can stop it by running

catt -d "NAME_OF_DEVICE" stop

At this point you’re all set with CATT, unless you want to set up some default settings. If you do, see the CATT Readme section here. It’s entirely optional.

Setup in HomeAssistant

Trusted Networks

Now that we have CATT set up, it’s time to get it working with HASS. By default CATT won’t work with HASS because HASS requires authentication and there’s no way to get a keyboard on a Hub. To get around this limitation, we’ll need to configure a Trusted Network in HASS (this is assuming you are running a version of HASS >0.89).

There’s a few ways to do this. You can whitelist your entire network range if you want, or you can do individual devices. You also can limit the users available. Personally I wanted to restrict things as much as possible so I only allowed the 3 Hubs I have to be considered to be on a trusted network and restricted the user to a special “Dashboard” account I created in HASS. So my configuration.yaml looks like this:

homeassistant:
# Note that this section is a subset of homeassistant: and is not tabbed all the way to the left like other things in configuration.yaml
  auth_providers:
    - type: trusted_networks
      trusted_networks:
        - IP_ADDRESS_1/32
        - IP_ADDRESS_2/32
        - IP_ADDRESS_3/32
      trusted_users:
        IP_ADDRESS_1: DASHBOARD_USER_ID
        IP_ADDRESS_2: DASHBOARD_USER_ID
        IP_ADDRESS_3: DASHBOARD_USER_ID
      allow_bypass_login: true
      # Note that bypass_login only works if one user is authorized

Under “trusted networks” I list out the 3 IP addresses with a /32 to limit the range to just that single IP. Under “trusted user” you list the IP address of the device and then the user_id(s) you want available. You can find the user_id if you go to the HASS configuration page, click “Users” and then click the username. You should see a line that says “ID” and a bunch of letters and numbers. Copy that.

Now that you have the trusted networks set up, reboot HASS and test it out to make sure it works. Find a page you want to cast (you will need to use the internal address and not any external URL you might use). So an example cast might be:

catt -d "NAME_OF_DEVICE" cast_site http://HASS_URL:PORT/lovelace/0

Replace HASS_URL with the IP address of your server and pot with the port. Also change “lovelace/0” to whatever page you want to cast

You should see a login screen on your Hub, but you’ll be able to select your dashboard user and login. You also can click the “Save login” button in the bottom right corner to avoid having to log in every time.

Automating

Assuming everything is working so far, now comes the fun part, automating! To use CATT in a HASS automation you need to use the Shell_Command integration. This is simply a line just like you would put in terminal to start up CATT. So for example:

shell_command:
  cast_weather: catt -d "NAME_OF_DEVICE" cast_site http://HASS_URL:PORT/lovelace/weather

Goes in your configuration.yaml file

You’ll need to restart HASS before the command is available. Once it is, you can invoke it using the shell_command service call. So using the above example, an automation might look like this:

- alias: "Good Morning Radar"
  trigger:
    platform: state
    entity_id: input_boolean.fuzzy_awake
    to: on
  action:
    - service: shell_command.cast_weather
    - delay: "00:01:00"
    - service: shell_command.cast_stop

Note that I also created a “cast_stop” shell command to stop the casting, and have it run 1 minute after the cast starts. Another way to do this is to create a bash script and call that instead. Here’s an example:

#!/bin/bash
/home/fuzzy/.local/bin/catt -d 'DEVICE_NAME' cast_site "URL"

sleep 1m

/home/fuzzy/.local/bin/catt -d 'DEVICE_NAME' stop

Adjust the paths, device name and URL as appropriate

Since I run HASS in Docker, I actually installed CATT on the server running HASS and then use SSH to remote out of the docker container to the server. Seems kinda meta but it works. But that’s outside the scope of this post (I plan on a post soon on how to restrict SSH’ing so it’s not a gaping security hole).

Conclusion

CATT is definitely a bit more complicated to set up that Home Assistant Cast but it has a few advantages. For one, it actually works (though I do hope that the HASS devs get it working consistently, its a really cool concept!). Also, since it’s all local you don’t need to make your HASS instance open to the outside world and it should work even if your internet is down.

Smart Home Hardware

Fri, 24 Apr 2020 02:27:00 +0000

Well now that I’ve talked about what software I run my smart home on, what about the hardware? Let’s dig in.

(Note: All links here do NOT contain any kind of affiliate link. They’re just direct links to the product)

Thermostat

I only have a single zone in my house so I went for an Ecobee 3. At the time I made my purchase, it was unique in that it had the ability to use remote sensors. These sensors can be placed in different rooms and then the thermostat can be told to include different sensors during different comfort profiles. So for example, during the day I don’t care what the temperature is in the bedrooms but I do care about the first floor, and vice versa at night. Nest later added similar functionality so this is no longer unique.

One major issue with Ecobee is their servers. There’s been so many server outages that I can’t even count. It’s almost a joke these days. There’s also the “Home IQ” reports that are comical in how delayed they can be. I can remember times when I’d log in to see and I literally have 3-4 months in a row without a report. To the point where beestat.io is what I would recommend people use instead of the Ecobee page.

Days like these aren’t uncommon

In fairness to Ecobee, things have gotten better. I haven’t seen nearly as many outages lately and I can view the Home IQ reports for the past several months as of writing this. Here’s the other thing: I rarely open the app or check my historical data, and I don’t think I’ve ever noticed a server outage. I have my thermostat programmed to follow a schedule and that continues to work even when there is a server outage.

Probably my favorite feature of the Ecobee is “vacation mode.” I use this to set the temperatures to hold at slightly warmer/colder temperatures (depending on the season) when I’m away. I use this heavily to adjust some of my home automations and I can set it via my calendar (works via HomeAssistant, you can also manually schedule in the app).

In hindsight, I might have gone with a Zwave thermostat if I was to do it again due to the server issues (and desire for local control). That being said, I do like some of the features that are built in to the Ecobee (sensors/vacation mode) and I haven’t really noticed server problems. Overall, I’m happy enough with my Ecobee.

Smart Home Hub

Ahhh the great Amazon Alexa v. Google Assistant debate. I’m not going to spend much time on this just because I think the decision is largely personal. For me, I started with all Echo Dots but recently got fed up with them. I found myself having to repeat commands frequently (sometimes many times) and often times with a result I didn’t want.

I did explore building my own smart assistant with some Raspberry Pis and Rhasspy. It worked but to achieve what I was after (whole home music system and voice assistant) was going to be 1) significantly more expensive (about $100 per set up) and 2) not quite as good as Alexa/Google. I’ve had a few Nest Hub’s (the small one with the screen) and really liked them, so after some discussions decided to rip the Echo Dots out and switch to Google.

Overall I’ve found the Minis to be superior. Much better recognition of commands and I really like the ability to cast music from my phone. I have hit a few issues like automating music playback but that’s partly due to my own personal use cases. I also wish I could turn on “brief mode” for every response instead of just when the device is in the same virtual “room” as whatever device I’m trying to control. IE right now if the device answering me is in the kitchen and so is the light, I get a “bing” in response, if the light is in the basement I get “Ok, turning on 3 lights in the basement.” Versus Alexa was always a “bing.” I also have some issues with integrating the Ecobee temperature sensors but it’s not a dealbreaker for me.

So really, try out both and see what works for you. In reality they’re pretty much comparable in terms of the supported integrations and such (with some exceptions I know).

ZWave

ZWave comprises a large majority of my smart home devices. I use it for all my light switches and most of my motion/door sensors. What I like about ZWave is that it uses it’s own spectrum so there’s very little chance of interference and every wired/plugged in device acts as a repeater so it’s fairly easy to build a robust and expansive ZWave network. It’s also locally controlled via a hub. For a hub, I use the Aeotec Z-Stick Gen 5. Haven’t had any problems with it, and highly recommend it.

The downside to ZWave is that it is a proprietary protocol that requires a license to make products. The result is that prices are higher than similar WiFi or Zigbee devices. For example, Zwave motion sensors are usually $30+ whereas Zigbee ones are $15-20.

Lights

For my lights, I pretty much only have GE switches. I do have a few Zooz ones and they’re OK, but overall I prefer the GE ones. The main drawback to the GE ones is that they 1) require a neutral wire (which most modern houses have but NOT ALL) and 2) they require an “add-on” switch when there are multiple switches on the same circuit (which are about $20/switch). That can make them an expensive option, especially when you look at both Zooz and Inovelli which have solutions that only require the “primary” switch to be smart and the other switches can be “dumb.” Both these solutions start coming out after I was mostly done installing the switches so in hindsight I would probably look more closely at Inovelli, but I’m still very pleased with my GE switches and don’t regret the decision all that much. The few Zooz switches I have work well but there are a few quirks that make them not a good fit for my use (mostly because I have so many GE switches that I just wanted consistent behavior across the board).

Basically every switch in my house at this point is “smart.” A lot of people (myself included) start off with smart light bulbs. And they work great. But there are 2 major issues with smart bulbs. 1) you (with a few exceptions) lose the ability to use the wall switches and 2) they can get $$$ quickly. If you have a room with 4+ lights, that’s easily $50+ in bulbs, versus $30 for a switch. Plus having to train guests/visitors to not use the switches is just….no.

Motion Sensors

I started out with Dome motion and door sensors and overall they work well. My gripes are that every now and then the door sensors don’t realize they’re closed when I open/close the door somewhat quickly and with the motion sensors the luminescence sensors work only in about 25% of the ones I have. Also, despite the fact you can configure the sensitivity of the motion sensors and they’re supposed to be “pet proof” my cats set them off all the time. Not a huge deal nor unique to the Dome sensors but….frustrating.

Locks/Garage

For a lock, I use the Schlage Connect and I LOVE it. I haven’t had to use my house key pretty much ever. When we go on vacation and need to let the pet sitters in I can turn on a door code that works for them (and is automatically turned on/off based on “vacation” mode). Family members also have key codes. And honestly it just works. Really all I can ask for in a door lock.

For the garage, I use the GoControl/Linear Zwave. Again, I love it. It works well, haven’t had any issues with it at all. Can’t really say much more than that. There are some other options out there that aren’t ZWave based but if you want locally controlled and have a ZWave network, I’d strongly encourage you to get the GoControl. And please please please please please STAY THE HELL AWAY FROM MyQ!!!

Zigbee

So this is a recent addition to my smart home. Zigbee is another protocol that is similar in some ways to ZWave in that it’s 1) locally controlled and 2) you need a hub. The major differences are that Zigbee is (I believe) an open standard (at least one without licensing fees) so it’s easier and cheaper to make products, and more importantly it uses the same spectrum as 2.4 GHz WiFi. Because it uses the same spectrum it’s possible that you’ll run into interference. If you want to learn more/read how to avoid interference, go read this article.

One of the more popular brands of Zigbee devices is the Xiaomi Aqara. There are a lot of different options here, from single click buttons to switches to magic six sided cubes. The best part about them is that they are incredibly cheap, especially if you order them on eBay where you can get them for about $10-15. Downside is the long shipping times and you can get them on Amazon in some cases for a few dollars more if you’re impatient. So far I’ve only used a few of the devices but I really like them. The main reason I decided to go down the Zigbee hole was because of the single button devices. With the death of the Amazon Dash buttons I was exploring other options and….they really didn’t exist. Zwave had some options but they were all $40+, and the only other options I could find were DIY which didn’t appeal to me.

To control my Zigbee devices, currently I’m using a Conbee II (running the deConz software). I tried using ZHA in HomeAssistant but had some issues. I’m also not sure I’m keeing the Conbee because I ordered a CC2531 dongle from Sonoff. This will supposedly work with Zigbee2MQTT. For $7 shipped I’m intrigued to see how well it works compared to the $40 Conbee II. However I’m still waiting for the CC2531 to arrive so can’t test it yet. Hoping it arrives soon, and maybe I’ll do an update on it once I get to play with it and make a final decision. For now, the Conbee works great.

Doorbell

Most of the smart things I get I end up very happy with because I research and think hard about what I get. I thought I’d done my homework in purchasing a Skybell HD. I got it on a Black Friday deal for like $130 and it was The Wirecutter’s pick for best smart doorbell. I installed it and everything was great. Best of all, there were no subscription fees (unlike Ring which was the only major competition at the time) and cloud storage for 7 days included. Great!

The shine did not last long. The motion sensing on the Skybell was utter garbage. Even at it’s lowest sensitivity it was picking up large trucks and school buses driving by my house (about 50 feet away). Then conversely it would miss when UPS/USPS/FedEx would drop off a package. I could understand how it could pick up a truck 50 feet away but not a person standing right in front of it. To top it off, when I did get notifications oftentimes they were minutes later. I kept hoping for software or firmware improvements (like setting detection zones) but they never came. And oftentimes when I’d try to load the stream I would just get a black picture. After 2 years, I was ready to move on.

Enter the Eufy 2k Doorbell. For $160 (usually on sale for $110ish) this is an amazing doorbell. The picture quality is really good, it’s fast to start streaming, and notifications are usually very quick. Also, it allows you to set detection zones AND it uses some kind of AI to recognize human faces, so you can customize the detection settings a bit. The result is that it’s probably 95% accurate in picking up a human approaching (5% is picking up my front porch flag and trees moving on particularly windy days).

In terms of storage there’s no cloud option and instead the storage is all local on a 4 GB built in SD card, which is both a pro and a con. It’s nice that the storage is local but I really wish there was a way to record via RTSP or back up to a NAS. I’m not so worried about someone stealing my doorbell cam (though that’s a legitimate concern) but more I just would prefer the storage to be off the device. That’s probably my main gripe (and the lack of HomeAssistant support but…that’s being worked on). Oh and you can’t use your existing doorbell, you need to use the little chime that you plug into a wall. Wish it used the existing bell but not a big deal to me.

Eufy also now has a battery powered version of the camera that saves footage to a Homebase (which is sort of like a NAS). It also apparently can tie directly into your existing chime AND wiring, so if I had to do it again I probably would go that route instead. I really like Eufy (sub-brand of Anker who I find to be highly reliable) and find their products to be good quality. Which leads me to….

Cameras

I won’t spend much time on this one because I’ll probably have more to say in a month or so. Right now I use Wyze v2 cams flashed with the RTSP firmware to get the feed into HomeAssistant. It’s….not great. The cams work well enough and the video quality is acceptable, but that’s when you’re using the app. The RTSP firmware has been in “beta” for over a year now and hasn’t seen an update in 6 months. There are tons of drop outs/disconnects that I just can’t rely on getting a picture consistently enough to do anything more than just checking the streams on occasion in HomeAssistant. I tried running automations off motion detection but just ended up frustrated. And I tried multiple NVR solutions (BlueIris, MotionEye, ZoneMinder, Shinobi…) and all had their issues. For a $25 camera I really can’t be mad but I do wish the RTSP firmware worked a little bit better. Oh, and the Google Home integration is equally frustrating in that it’s slow to work (if it ever actually loads the feed) and incredibly laggy to the point of uselessness.

So when Eufy recently announced their new indoor 2k cameras I was immediately intrigued. They’re only available for pre-order at the moment (though they’re ~30% off which brings them down to Wyze level pricing. Crucially for me is that they support RTSP natively. I’m curious to see how well they do but if they’re anything like the doorbell camera I think they’ll work great. I ordered several cameras to basically replace every Wyze camera I have and I’ll post an update/review once I receive them (currently supposed to be released at the end of May). So come back for more later.

Other Sensors

As I said above, most of my door and motion sensors are Zwave. A few months ago I picked up some Wyze contact and motion sensors and overall I’m REALLY impressed by them. They are incredibly cheap (you can get a 4 motion sensors or contact sensors for ~$20) and are very responsive. They use a proprietary 900 MHz frequency so no interference and VERY long range. The downside is you need yet another hub (comes with the starter pack). Normally you plug the hub into the back of a Wyze camera but some smart people reverse engineered things and you can now get them imported into HomeAssistant with the hub plugged directly into a computer. No Wyze Cam necessary. The only real drawback I’ve found with the motion sensors is that they have a 40 second cool down which is a bit longer than I’d like (and it’s not adjustable). Not a major problem in most situations but a shorter cool down would be nice.

Conclusion

Phew, that ended up longer than I was expecting it to be. Any post about smart home hardware is going to be highly personalized to the individual user, but I thought it might be useful to see a fairly wide overview of what is out there and what kind of devices are useful. And it’s fun for me to review some of the stuff I’ve tried and not liked and some of the stuff I still love 2+ years later.

What software do I run?

Wed, 22 Apr 2020 01:55:00 +0000

Maybe the first place to start is: what software am I running to control my smart home?

Surprisingly, a lot.

HomeAssistant

First, and most importantly, is Home-Assistant, or HASS. HASS is…incredible. HASS can literally integrate with thousand of devices and services (1,500+ as of writing this) and it’s continually increasing. Not to say that HASS is perfect (it’s not). There are still a lot of growing pains as it’s not 1.0 yet, which means lots of breaking changes between updates and some of them aren’t popular. I still think it’s the best piece of software for running a smart home given it’s popularity, ease of installation, and helpful userbase. I probably won’t do any kind of “How to Install HASS” as that’s well covered elsewhere (and again, kinda everchanging at the moment) but I probably will do some posts about the various things I integrate with HASS and how I made those decisions.

NodeRed

Next is Node-Red. The best way to think about Node-Red is that it’s the “brains” running the show. HASS provides the connections to my various devices and their state and then Node-Red tells HASS what to do. Node-Red is a visual automation editor that I have found to be far superior to the built in HASS automation editor. I started out with HASS as the automation engine and it worked very well for me. But I grew tired of repeating lines of code over and over when I’d add things and debugging was a pain. Node-Red is much easier in my opinion to 1) see what you’re doing, 2) easily reuse what you’ve done before (I love using links), and 3) debug your automations to see if there are problems and where they are. At this point I have very few automations left in HASS (mainly those I just can’t figure out how to recreate or am to lazy to deal with).

Example of Node-Red in action

The above automation replaced a few hundred lines of code and controls my alarm system. I have some other examples I’ll explore later, but I would highly recommend someone starting out with HASS to explore Node-Red as well.

Other Software

Just a quick rundown of other software I run. MQTT (Message Queuing Telemetry Transport) is an absolutely essential as it’s an incredibly versatile tool. I use it a lot of interconnect various devices and services. Basically you create a MQTT broker/server and then various clients can connect to it to both send and read/receive messages on different topics. Sounds complicated but once you dig into it a bit you’ll see how simple and flexible it can be. Personally I use Mosquitto but there are a few other options out there.

I’ll have a larger post on hardware at some point, but recently I picked up a Zigbee stick (Conbee II) and to control the various Zigbee devices I’ve picked up I use deConz. Again, more on hardware later.

Database-wise I mostly use Mariadb, but also have an InfluxDB that feeds HASS data into Grafana for some pretty graphs. Not much to say here, a database is kind of a database…

Other odds and ends that are tangentially related to smart home things are MotionEye for recording my various Wyze Cams running RTSP (more on these later). Nginx is my reverse proxy of choice (I tried Apache and Traefik but I have nginx working so I’m following the “if it ain’t broke don’t fix it” mantra).

Docker

Everything described above is run via Docker and Docker-Compose. Docker makes things so much simpler to set up, configure, and run. Basically everything installs inside of a container so everything should “just work.” If it doesn’t work, I can easily delete the container/image with minimal system impacts (and best of all, very few lingering files/folders/other cruft). I end up trying a lot of things out and really like using Docker to test things out for that reason. In fact, if there’s NOT a Docker container available I’m less likely to want to try the software out these days. The result is that there’s very little that runs outside of Docker for me on my server.

Finally, speaking of my server, I’m running Ubuntu 18.04 LTS (soon to be upgraded to 20.04 LTS once the .1 patch comes out). Ubuntu works well and is well supported so again it “just works.” Again part of the benefit of running everything in Docker is that the underlying OS doesn’t matter a ton so use what works for you.

Conclusion

So that’s what runs my smart house. Probably an overly simplistic overview but hope it’s helpful to someone to see. Next up I’ll probably cover the hardware I use, before I start branching off a bit to some of the more nuts and bolts things that I want to write to document/write up.

Till next time.

First

Wed, 15 Apr 2020 18:10:20 +0000

First blog post. Wheeeeeee.

I’m still not 100% sure what I’ll use this for. But I wanted to start trying to document some of the smart home/tech things I’m doing so I can 1) maybe help someone else out who is having a similar problem and 2) make a resource for myself in case I have to rebuild everything (really hoping that never happens!).

So expect some things about smart homes, some things about general Linux, more things about smart homes….maybe some other random things. We’ll just have to see won’t we?

Posts on Random Musings

MXroute Email Aliases - A SimpleLogin Replacement

SimpleLogin

MXroute

MXroute Aliases

Conclusion

Printing HomeBox QR Code Labels to HomeAssistant over MQTT

Homebox Setup

To HomeAssistant!

Final Steps

Migrating to PostgreSQL

How to Enable IPv6 on OPNSense for Matter Support

IPv4 v. IPv6

OPNsense Configuration

Omada Configuration

Configuring Proxmox

Matter Docker Container

Conclusion

Setting up Pocket-ID with Caddy-Security

Setting up Pocket-ID

Configuring Pocket-ID

Apps that Support OIDC

Apps that Don’t Support OIDC

Caddyfile

Explanation/Things I Learned

Restricting to Groups

Bypassing Auth

Further Reading

The Amazing Arc Pro A40 and Problems with Coral TPU Passthrough

Transcoding Setup

Coral TPU over USB

Arc Pro Transcoding Performance

How to Send an Snapshot of a Traffic Map to Your Phone

Install Splinter

Install Selenium

Google Maps Setup

Waze Setup

Putting it All Together

HomeAssistant

2024 Homelab Remodel

No More Drone

Goodbye Synology

LNXLink

Conclusion

Bye Bye Minio, Hello Restic Rest Server!

Minio Issues

Minio Alternative - Restic Rest Server

Final Thoughts

Permanently Mounting Cameras without Damaging House

HomeAssistant App and TinyCam - Powerful Combination!

MyQ - Local Garage Door Control

New House - New Smart Home Hardware!

Thermostat

Smart Home Hub

Zwave

Zigbee

Sound Machines

Doorbell/Cameras

Other/Misc.

Conclusion

New Comment System - Giscus

Revisiting BlueIris, Deepstack and Homeassistant

Hardware

Intel gvt-g

Deepstack Setup

Initial Configuration

Alert Settings

HomeAssistant Integration

Conclusion

Deconz v. Zigbee2MQTT

deConz

Zigbee2MQTT

Conclusion

Valetudo on a Dreametech Vacuum

Hardware

D9

Z10

Installation

Conclusion

Ioniq 5 - My First EV