Using rootful podman-compose on Centos9

In the last weeks I updated my home server from Centos8 to Centos 9 stream. As part of this I also switched from docker to podman. This was no problem for single container but in case of multiple container working together - aka are composed - it wont work as expected. 

E.g in a setup of two container in a pod - app and db - the app can't resolve (and reach) the db. 

After multiple trial and error setups, crawling, reading through the web - filter out obsolete tips etc. I switch to "netavark" - (see https://github.com/containers/podman-compose/issues/455), be aware that the mentioned command podman system reset --force will remove all volumes (Yes, I did have a backup.)

In the end - all does not help ... I was about to give up. But then I found a hint how to look in the dns config of the container:

$ more /run/containers/networks/aardvark-dns/gogs_default
10.89.0.1
9951847ba473d97b6bf1e834b490 10.89.0.4 gogs_db,db,9951847ba473
6b06aee614d7584d32f3912e3b2c 10.89.0.5 gogs_app,app,6b06aee614d7

Ok, this mean the aardvark-dns is configured right. Next I look in the logs:

$ journalctl -t aardvark-dns
21:06:57 s...de aardvark-dns[1998]: Received SIGHUP will refresh servers: 1
21:06:57 s...de aardvark-dns[1998]: Unable to start server unable to start CoreDns server: \
                                    Address already in use (os error 98)

Looks like the aardvark-dns is in conflict with my name server runing on the same maschine.

$ ps 1998
 PID TIME COMMAND
1998 0:00 /usr/libexec/podman/aardvark-dns --config /run/containers/networks/aardvark-dns -p 53 run

Interesting - it is running on port 53 - like bind.

Finally I found out how to configure the port aardvark is using: it is a further entry in the file where I already switch to netavark.

$ more /etc/containers/containers.conf
[network]

# Explicitly use netavark. 
# See https://github.com/containers/podman-compose/issues/455
network_backend = "netavark"

dns_bind_port = 5533

Restart ... it runs.

Update: Display TTN-Gateway status

After migrating my TTN gateway to the new The Things Stack Community Edition (see previous blog post) my script to display the gateway status stopped working. The reason behind was, that the new stack does not provide the noc api which was used in the script.

After searching in the TTN forum I found the thread New API for gateway mapping, status and info. Using this API I just need to change one line in my script and now it works again.

-  cmd = "curl -s http://noc.thethingsnetwork.org:8085/api/v2/gateways/eui-b827ebfffe06902a | jq -r '.timestamp'"

+  cmd = "curl -s https://mapper.packetbroker.net/api/v2/gateways/netID=000013,tenantID=ttn,id=eui-b827ebfffe06902a | jq -r '.updatedAt'"

The updated script is available in the github repo.

Migrate TTN-Gateway to The Things Stack Community Edition (aka V3)

 I just migrate my TTN-gateway to the new The Things Stack Community Edition (aka V3). 

I followed the instructions I found in the thread on GitHub.

1. Create a new gateway on the things network (link)
2. Change the server url in your local_conf.json to eu1.cloud.thethings.network
3. Restart the gateway

Unfortunately the new stack does not provide a NOC api - so my script to output the gateway status to a OLED stop working. I will invest into this next.

Photovoltaic dashboard with Grafana

I have a SolarEdge Inverter which is monitored by a energy meter (SolarEdge SE-MTR-3Y-400V-A). The energy meter gets measurements from the inverter and measures voltage and current at the house entry point.

SolarEdge provide a monitoring service and a mobile app to monitor the system. An example is shown here:

To be able to monitor and evaluate the measurements by myself I use the following applications:

• mbmd - to read the measurements from the inverter and energy meter
• nodered - to calculate some derived values 
• mqtt - to "transport" the measurements between the apps
• telegraf - to write the measurements into influxdb
• influxdb - to store the measurements
• grafana - to display the measurements

The result looks like this (a snapshot is available here):

Configuration 

mbmd

The mbmd reads the measurements from the inverter and (via the inverter) from the energy meter.

pi@racknode:/etc $ more mbmd.yaml
# REST api, use 127.0.0.1 to restrict to localhost
api: 0.0.0.0:8080

# mqtt config
mqtt:
  broker: my.mqtt.server:1883
  topic: mbmd
#  user:
#  password:
  clientid: mbmd
  qos: 0
  homie: ""
  # homie: homie


# adapters are referenced by device
adapters:
- device: inverter.ip.addr:1502

# list of devices
devices:
- name: inverter
  type: SE
  id: 1
  subdevice: 0 # use subdevice to access SunSpec subdevices
- name: powermeter
  type: SE
  id: 1
  subdevice: 1 # use subdevice to access SunSpec subdevices

Node-Red

To calculate the "self consumption" and split the measured power in Import/Export I create a Node-Red flow:

Here is the corresponding flow code.

Telegraf

The telegraf is configured to transfer the mqtt messages to the influxdb. 

The config is available here.

Grafana

A snapshot of the dashboard is available here and also the dashboard.json is available.

HP MFP M754DN Fix "Scan to Network" error

My multifunction printer HP MFP M754DN has the ability to scan to a network folder. The target folder is provided by a samba server running CentOS8. It works with out problem but stops to be able to access the network folder some weeks ago. 

Testing the connection on the webgui results in the message: "Cannot connect to the network folder. Ensure the shared folder name is correct."

To fix it I have added the following lines in the smb.conf file on the samba server.

server min protocol = NT1    
min protocol = NT1

A further analysis of my last update activities showed that this was related to the update of the samba package from samba-4.10.4-101.el8_1.x86_64 to samba-4.11.2-13.el8.x86_64. 

According to Samba 4.11 WhatsNew the "SMB1 is disabled by default" now - and the HP printer unfortunately uses only SMB1. With the mentioned settings you enabled the SMB1 protocol again.

TTN-Gateway in a box

During the last days I worked on the installation of my TTN-Gateway into a case. This is the result:

Beside the LoraWAN hardware I have installed an OLED display and 3 sensors (1x Si7021, 2x DS18B20). The OLED display is connected via I2C, the sensors via 1-wire and I2C.

The sensor data is read by Python3 Script and published with MQTT. The OLED is also controlled by a Python3 script. Both scripts run as systemd-service.

The software is available on github.

Little helper: mwc = more-without-comment

While configuring my linux nodes I found my alias mwc very helpful:

I used it to show the relevant (= uncommented and non-empty) lines in typical linux config files.

Updated inadyn-mt package for CentOS 8

I have updated my inadyn-mt rpm/srpm package for CentOS 8. It is based on inadyn-mt v02.28.10.

• inadyn-mt-2.28.10-4.el8.x86_64.rpm (RPM)
• inadyn-mt-2.28.10-4.el8.src.rpm (SRPM)

I follow again the steps I described here, but started with the current source rpm inadyn-mt-2.28.10-4.fc31.src.rpm

Setting up a Zigbee2mqtt gateway

Here are the steps I performed to setup a Zigbee2mqtt gateway using a CC2530+CC2591 board and a RaspberryPI 3.

• My RaspberryPI 3 is running

  pi@racknode:~ $ uname -a
  Linux racknode 4.19.42-v7+ #1219 SMP Tue May 14 21:20:58 BST 2019 armv7l GNU/Linux
  pi@racknode:~ $ hostnamectl
     Static hostname: racknode
           Icon name: computer
          Machine ID: confidential
             Boot ID: confidential
    Operating System: Raspbian GNU/Linux 9 (stretch)
              Kernel: Linux 4.19.42-v7+
        Architecture: arm

• My Zigbee board:
  
  
  
  
  

The steps can be structured in three groups:

1. Flash the Z-Stack coordinator firmware on the Zigbee board
2. Install zigbee2mqtt to run on the Raspberry Pi
3. Setup a daemon to run zigbee2mqtt

Flash the Z-Stack coordinator firmware on the Zigbee board

• Connect the Zigbee board with the Raspberry (Note: this is the connection schema for flashing the firmware, while normal operation another connection schema is used.)
  
  

  Name	Raspi	CC2530+CC2591*
  3,3V/VCC	Pin01	P12 Pin02
  GND	Pin39	P12 Pin01
  RESET	Pin35	P12 Pin03
  DC	Pin36	P14 Pin04
  DD	Pin38	P14 Pin05

  *) P12 - 12pin connector, P14 - 14 pin connector
  
  
  
  
  
  
  
  
  
  
• Install the wiringPI library
  

  sudo apt-get install wiringpi

• Install git

  pi@racknode:~ $ sudo apt-get install git
  

• Clone the flasher repo

  git clone https://github.com/jmichault/flash_cc2531.git

• Test the connection

  pi@racknode:~/flash_cc2531 $ ./cc_chipid
    ID = a524.

• Get the firmware - as mentioned above I need the CC2530+CC2591 version. See https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_Home_1.2/bin/default for the correct version.

  pi@racknode:~/flash_cc2531 $ wget https://github.com/Koenkk/Z-Stack-firmware/raw/master/
  coordinator/Z-Stack_Home_1.2/bin/default/CC2530_CC2591_DEFAULT_20190608.zip

• Extract the archive

  pi@racknode:~/flash_cc2531 $ unzip CC2530_CC2591_DEFAULT_20190608.zip
  Archive:  CC2530_CC2591_DEFAULT_20190608.zip
    inflating: CC2530ZNP-Prod.hex
    inflating: CC2530ZNP-Prod.bin
  pi@racknode:~/flash_cc2531 $

• Erase the CC2531 CC2530

  pi@racknode:~/flash_cc2531 $ ./cc_erase
    ID = a524.
    erase result = 00a6.
  pi@racknode:~/flash_cc2531 $

• Write the new firmware (takes about 2-3 minutes)

  pi@racknode:~/flash_cc2531 $ ./cc_write CC2530ZNP-Prod.hex
    ID = a524.
    reading line 15490.
    file loaded (15497 lines read).
  writing page 128/128.
  verifying page 128/128.
   flash OK.
  pi@racknode:~/flash_cc2531 $ cd ..
  pi@racknode:~ $

Install zigbee2mqtt to run on the Raspberry Pi

Steps based on

• Connecting CC2530
• Running zigbee2mqtt

My steps were

• Connect the Zigbee board with the Raspberry for operation mode
  
  

  Name	Raspi	CC2530+CC2591*
  3,3V/VCC	Pin01	P12 Pin02
  GND	Pin39	P12 Pin01
  TX (R→Z)	Pin08	P12 Pin07
  RX (R←Z)	Pin10	P12 Pin08

  *) P12 - 12pin connector
  
  
  
  
  
  
  
  
  
• Enable UART and disable UART use for Bluetooth by adding following lines to /boot/config.txt

  pi@racknode:~ $ sudo vi /boot/config.txt
  
  enable_uart=1
  dtoverlay=pi3-disable-bt
  

• Disable modem system

  sudo systemctl disable hciuart

• Remove the serial console entry from /boot/cmdline.txt
  Before

  pi@racknode:~ $ cat /boot/cmdline.txt
  dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=...
  

  After

  pi@racknode:~ $ cat /boot/cmdline.txt
  dwc_otg.lpm_enable=0 console=tty1 root=...
  

• Reboot the Raspberry
• Installing the node.js 10.0 repo and install node.js

  pi@racknode:~ $ sudo curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -

  ... and install node.js

  pi@racknode:~ $ sudo apt-get install -y nodejs

• Clone the zigbee2mqtt repo

  pi@racknode:~ $ sudo git clone https://github.com/Koenkk/zigbee2mqtt.git /opt/zigbee2mqtt

  ... and change own to pi

  pi@racknode:~ $ sudo chown -R pi:pi /opt/zigbee2mqtt

• Install the dependencies

  pi@racknode:~ $ cd /opt/zigbee2mqtt
  pi@racknode:/opt/zigbee2mqtt $ npm ci
  
  prebuild-install WARN install No prebuilt binaries found 
  (target=10.17.0 runtime=node arch=arm libc= platform=linux)
  make: Entering directory '/opt/zigbee2mqtt/node_modules/zigbee-herdsman/
  node_modules/@serialport/bindings/build'
    CXX(target) Release/obj.target/bindings/src/serialport.o
    CXX(target) Release/obj.target/bindings/src/serialport_unix.o
    CXX(target) Release/obj.target/bindings/src/poller.o
    CXX(target) Release/obj.target/bindings/src/serialport_linux.o
    SOLINK_MODULE(target) Release/obj.target/bindings.node
    COPY Release/bindings.node
  make: Leaving directory '/opt/zigbee2mqtt/node_modules/zigbee-herdsman/
  node_modules/@serialport/bindings/build'
  
  > serialport@8.0.1 postinstall /opt/zigbee2mqtt/node_modules/zigbee-herdsman/
  node_modules/serialport
  > node thank-you.js
  
  Thank you for using serialport!
  If you rely on this package, please consider supporting our open collective:
  > https://opencollective.com/serialport/donate
  
  
  npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/
  zigbee-herdsman/node_modules/fsevents):
  npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for 
  fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm"})
  npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/
  fsevents):
  npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for 
  fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm"})
  
  added 1356 packages from 856 contributors and audited 877179 packages in 220.282s
  found 3 high severity vulnerabilities
    run `npm audit fix` to fix them, or `npm audit` for details

• Fix the mentioned issues

  pi@racknode:/opt/zigbee2mqtt $ npm audit fix
  npm WARN tsutils@3.17.1 requires a peer of typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || ↵
  >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta but ↵
  none is installed. You must install peer dependencies yourself.
  npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/↵
  zigbee-herdsman/node_modules/fsevents):
  npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: ↵
  wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm"})
  npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents):
  npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: ↵
  wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm"})
  
  updated 2 packages in 52.752s
  fixed 3 of 3 vulnerabilities in 877319 scanned packages
  pi@racknode:/opt/zigbee2mqtt $
  

• Enter your configuration in /opt/zigbee2mqtt/data/configuration.yaml

  vi data/configuration.yaml

  Enter your settings 

  # MQTT settings
  mqtt:
    # MQTT base topic for zigbee2mqtt MQTT messages
    base_topic: zigbee2mqtt
    # MQTT server URL
    server: 'mqtt://localhost'
    # MQTT server authentication, uncomment if required:
    # user: my_user
    # password: my_password
  # CC2590 is connected to serial line
  serial:
    port: /dev/ttyAMA0
  advanced:
    baudrate: 115200
    rtscts: false 
  

• Start zigbee2mqtt

  pi@racknode:/opt/zigbee2mqtt $ npm start
  
  > zigbee2mqtt@1.7.1 start /opt/zigbee2mqtt
  > node index.js
  
  Logging to directory: '/opt/zigbee2mqtt/data/log/2019-11-30.16-53-43'
  Starting zigbee2mqtt version 1.7.1 (commit #b459c35)
  Starting zigbee-herdsman...
  zigbee-herdsman started
  Coordinator firmware version: '{"type":"zStack12","meta":{"transportrev":2,↵
  "product":0,"majorrel":2,"minorrel":6,"maintrel":3,"revision":20190608}}'
  Currently 0 devices are joined:
  `permit_join` set to  `true` in configuration.yaml.
  Allowing new devices to join.
  Set `permit_join` to `false` once you joined all devices.
  Zigbee: allowing new devices to join.
  Connecting to MQTT server at mqtt://xxx.xxx.xxx
  Connected to MQTT server
  MQTT publish: topic 'zigbee/bridge/state', payload 'online'
  MQTT publish: topic 'zigbee/bridge/config', payload '{"version":"1.7.1",↵
  "commit":"b459c35","coordinator":{"type":"zStack12","meta":{"transportrev":2,↵
  "product":0,"majorrel":2,"minorrel":6,"maintrel":3,"revision":20190608}},↵
  "log_level":"info","permit_join":true}'

Setup a daemon to run zigbee2mqtt

based on zigbee2mqtt - 5. (Optional) Running as a daemon with systemctl

• Create a zigbee2mqtt service

  sudo vi /etc/systemd/system/zigbee2mqtt.service

  enter

  [Unit]
  Description=zigbee2mqtt
  After=network.target
  
  [Service]
  ExecStart=/usr/bin/npm start
  WorkingDirectory=/opt/zigbee2mqtt
  StandardOutput=inherit
  StandardError=inherit
  Restart=always
  User=pi
  
  [Install]
  WantedBy=multi-user.target
  

• Verify that it works

  pi@racknode:/opt/zigbee2mqtt $ sudo systemctl start zigbee2mqtt
  pi@racknode:/opt/zigbee2mqtt $ sudo systemctl status zigbee2mqtt
  ● zigbee2mqtt.service - zigbee2mqtt
     Loaded: loaded (/etc/systemd/system/zigbee2mqtt.service; disabled; ↵
     vendor preset: enabled)

•    Active: active (running) since Tue 2019-12-03 08:35:46 CET; 7s ago
   Main PID: 626 (npm)
      Tasks: 23 (limit: 4915)
     CGroup: /system.slice/zigbee2mqtt.service
             ├─626 npm
             ├─643 sh -c node index.js
             └─644 node index.js
  
  Dec 03 08:35:46 racknode systemd[1]: Started zigbee2mqtt.
  Dec 03 08:35:50 racknode npm[626]: > zigbee2mqtt@1.7.1 start /opt/zigbee2mqtt
  Dec 03 08:35:50 racknode npm[626]: > node index.js
  Dec 03 08:35:53 racknode npm[626]: [..] Logging to directory: '/opt/zigb
  Dec 03 08:35:53 racknode npm[626]: [..] Starting zigbee2mqtt version 1.7
  Dec 03 08:35:53 racknode npm[626]: [..] Starting zigbee-herdsman...

• Configure the autostart of the service

  pi@racknode:/opt/zigbee2mqtt $ sudo systemctl enable zigbee2mqtt.service
  Created symlink /etc/systemd/system/multi-user.target.wants/zigbee2mqtt.service ↵
  → /etc/systemd/system/zigbee2mqtt.service.

• Reboot the Raspberry to verify your setup

  pi@racknode:/opt/zigbee2mqtt $ sudo reboot
  Connection to racknode closed by remote host.
  Connection to racknode closed.
  
  $ ssh pi@racknode
  Linux racknode 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l
  
  The programs included with the Debian GNU/Linux system are free software;
  the exact distribution terms for each program are described in the
  individual files in /usr/share/doc/*/copyright.
  
  Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
  permitted by applicable law.
  Last login: Tue Dec  3 08:42:26 2019 from 10.0.0.171
  pi@racknode:~ $ sudo journalctl -u zigbee2mqtt.service -f
  -- Logs begin at Thu 2016-11-03 18:16:43 CET. --
  [..] Started zigbee2mqtt.
  [..] > zigbee2mqtt@1.7.1 start /opt/zigbee2mqtt
  [..] > node index.js
  [..] Logging to directory: '/opt/zigbee2mqtt/data/log/2019-12-03.08-44-14'
  [..] Starting zigbee2mqtt version 1.7.1 (commit #b459c35)
  [..] Starting zigbee-herdsman...
  [..] zigbee-herdsman started
  [..] Coordinator firmware version: '{"type":"zStack12","meta":{"transportrev":2,↵
       "product":0,"majorrel":2,"minorrel":6,"maintrel":3,"revision":20190608}}'
  [..] Currently 1 devices are joined:
  [..] 0x00158d00029acb06 (0x00158d00029acb06): MFKZQ01LM - Xiaomi Mi/Aqara smart ↵
       home cube (EndDevice)
  [..] `permit_join` set to  `true` in configuration.yaml.
  [..] Allowing new devices to join.
  [..] Set `permit_join` to `false` once you joined all devices.
  [..] Zigbee: allowing new devices to join.
  [..] Connecting to MQTT server at mqtt://mqtt.thomo.de
  [..] Connected to MQTT server
  [..] MQTT publish: topic 'zigbee/bridge/state', payload 'online'
  [..] MQTT publish: topic 'zigbee/0x00158d00029acb06', payload '{"battery":31,↵
       "voltage":2825,"linkquality":18,"angle":10.55,"side":3}'
  [..] MQTT publish: topic 'zigbee/bridge/config', payload '{"version":"1.7.1",↵
       "commit":"b459c35","coordinator":{"type":"zStack12","meta":{"transportrev":2,↵
       "product":0,"majorrel":2,"minorrel":6,"maintrel":3,"revision":20190608}},↵
       "log_level":"info","permit_join":true}'
  ^C
  pi@racknode:~ $

Links

• https://www.zigbee2mqtt.io/

Update

2020-02-15: Fix error in flash connection table and picture.

Setup TTN Gateway using Raspi3 with RAK831 and converter board

Finally I was able to put my TTN Gateway into operation.

The first attempt a few month ago failed and therefore the project was on hold until now. An article in make 03/2019 gave me the impetus for a new attempt.

My essential setup steps:

1. Put the parts together.
   Caution: Do not power up the RAK831 card without connecting the antenna.
2. Prepare a SDCard with a Raspberry Image - I use "Raspbian Buster with desktop" (Release date: 2019-09-26, Kernel 4.19)
3. Perform a general update
   

   sudo apt-get update && sudo apt-get upgrade

4. Enable SPI using raspi-config
5. Clone the ttn gateway repo
   

   git clone https://github.com/ttn-zh/ic880a-gateway.git

6. Run the setup script
   

   cd ic880a-gateway
   sudo ./install.sh

7. Register the gateway on TTN website

After the restart I check the service status:

pi@ttn-gateway:~ $ systemctl status ttn-gateway
● ttn-gateway.service - The Things Network Gateway
   Loaded: loaded (/lib/systemd/system/ttn-gateway.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Tue 2019-10-08 14:05:42 CEST; 779ms ago
  Process: 346 ExecStart=/opt/ttn-gateway/bin/start.sh (code=exited, status=1/FAILURE)
 Main PID: 346 (code=exited, status=1/FAILURE)

ttn-gw ttn-gateway[346]: INFO: Statusstream data is enabled
ttn-gw ttn-gateway[346]: INFO: Beacon is disabled
ttn-gw ttn-gateway[346]: INFO: Monitor is disabled
ttn-gw ttn-gateway[346]: INFO: Contact email configured to "my email"
ttn-gw ttn-gateway[346]: INFO: Description configured to "ttn-rak831"
ttn-gw ttn-gateway[346]: INFO: Successfully contacted server router.eu.thethings.network
ttn-gw ttn-gateway[346]: INFO: [main] Starting the concentrator
ttn-gw ttn-gateway[346]: ERROR: [main] failed to start the concentrator
ttn-gw systemd[1]: ttn-gateway.service: Main process exited, code=exited, status=1/FAILURE
ttn-gw systemd[1]: ttn-gateway.service: Failed with result 'exit-code'.

Damn - the same problem as at first try! :-(

An issue thread on github list some possible reasons, among other "the wiring might be wrong". You may also interpret it as "other then expected or as default" and this was the key. On another page about the setup I found the crucial clue: The converter board uses a different reset pin (it use pin 17 instead of 25).

After changing the corresponding setting in /opt/ttn-gateway/bin/start.sh

#! /bin/bash

# Reset iC880a PIN
# SX1301_RESET_BCM_PIN=25
SX1301_RESET_BCM_PIN=17
echo "$SX1301_RESET_BCM_PIN"  > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio$SX1301_RESET_BCM_PIN/direction

... it worked as expected. And I see the status "connected" on the TTN console (on TTN website).

To enable the GPS module I have to:

1. Disable the serial login shell but keep the serial interface enabled - via raspi-config - interface setup.
2. add the following lines to /opt/ttn-gateway/bin/local_conf.json
   

   "gps_tty_path": "/dev/ttyS0",
   "fake_gps": false,
   

3. Reboot and check if it works (it may take some minutes to fetch the GPS position)

   pi@ttn-gateway:/opt/ttn-gateway/bin $ sudo tail -f /var/log/syslog | grep -i gps
   Oct  8 17:55:36 ...: INFO: GPS serial port path is configured to "/dev/ttyS0"
   Oct  8 17:57:39 ...: INFO: GPS is enabled
   Oct  8 17:57:39 ...: INFO: Using real GPS if available.
   Oct  8 17:57:39 ...: INFO: [main] TTY port /dev/ttyS0 open for GPS synchronization
   Oct  8 17:57:39 ...: INFO: GPS thread activated.
   Oct  8 17:57:39 ...: WARNING: [gps] GPS out of sync, keeping previous time reference
   Oct  8 17:57:39 ...: WARNING: [gps] GPS out of sync, keeping previous time reference
   Oct  8 17:57:39 ...: WARNING: [gps] GPS out of sync, keeping previous time reference
   Oct  8 17:57:39 ...: WARNING: [gps] GPS out of sync, keeping previous time reference
   Oct  8 17:57:39 ...: ### [GPS] ###
   Oct  8 17:57:39 ...: # Valid gps time reference (age: 0 sec)
   Oct  8 17:57:39 ...: # System GPS coordinates: latitude 50.94322, longitude 7.37443, altitude 272 m
   Oct  8 17:57:39 ...: ### [GPS] ###
   Oct  8 17:57:39 ...: # Valid gps time reference (age: 0 sec)
   Oct  8 17:57:39 ...: # System GPS coordinates: latitude 50.94322, longitude 7.37443, altitude 274 m
   

Used hardware:

Raspi3

The converter board with GPS support

RAK831 module

On the left side: 

top - the TTN antenna

bottom - the GPS module (optional)

Connection of the GPS signal

Update: The wiring of the converter board is available on the RAK website.

Updated inadyn-mt package for CentOS7

I have updated my inadyn-mt rpm/srpm package for CentOS7. It is based on inadyn-mt v02.28.06.

• inadyn-mt-2.28.06-2.el7.centos.x86_64.rpm (RPM)
• inadyn-mt-2.28.06-2.el7.centos.src.rpm (SRPM)

Setup/Configuration for another Magic Mirror

Note: Unfortunately, a glitch from the TV, which sometimes occurred before, seems now to be manifested. The TV doesn't power on anymore, neither using remote control nor buttons. I couldn't find a faulty component on TV circuit board. I will/have to stop this project. However, I have learned a lot, so it was not wasted time. 

General setup

• Raspberry PI 2
• SEG ArtColor TV 
• PIR module (GPIO 17, 18), IR transmitter (GPIO 4)

The PIR module will detect motion. If a motion is sense the TV will be switched on via the IR transmitter (simulation of IR remote). After a predefined timeout periode without further motion the TV will switched off (to standby).

Install and enable lirc

• install lirc

  sudo apt-get install lirc

• enable lirc module - edit /boot/config.txt

  dtoverlay=lirc-rpi,gpio_out_pin=4,gpio_in_pin=22

• configure lirc - edit /etc/lirc/hardware.conf

  LIRCD_ARGS=""
  ...
  DRIVER="default"
  DEVICE="/dev/lirc0"
  MODULES="lirc_rpi"
  

Setup IR command to power on/off the TV

• learn IR command

  irrecord -d /dev/lirc0 ~/lircd.conf

• copy new file

  sudo cp ~/lircd.conf /etc/lirc/lircd.conf

• edit the name parameter in /etc/lirc/lircd.conf I choose magicmirror
• start lirc

  pi@mamomami /etc/lirc $ sudo /etc/init.d/lirc start
  [ ok ] Loading LIRC modules:.
  [ ok ] Starting remote control daemon(s) : LIRC :.

• check if IR command is available

  pi@mamomami /etc/lirc $ irsend LIST "magicmirror" ""
  irsend: 00000000000043bc KEY_POWER
  

• run lirc service on start up

  sudo update-rc.d lirc defaults

Ansible: using numbered backreference followed by digit in lineinfile

While playing around with Ansible I've stumbled over a problem with backreferences followed by a digit.

The problem

I've want to modify some lines in a file like this:

# an example
something=xxxx

Lets assume I want to update the xxxx with some values.

A possible ansible playbook may look as follows:

---
- hosts: localhost
  tasks:
  - name: create file
    copy: src=./demo dest=/tmp/demo

  - name: update xxxx
    lineinfile:
      dest: /tmp/demo
      regexp: '^(.*)xxxx'
      line:   '\1{{ansible_date_time.year}}'
      backrefs: yes
      state: present

The update task "update xxxx" should inspect the file, search for the line with "xxxx" and replace the "xxxx" while keeping the part in front of it.
But the result is:

# an example
P16

The problem is that the backreference \1 is not resolved as expected but merged with (some) of the year digits. This results in the "P".

Solution

To solve it you have to use named groups. A named group is defined by (?P<name>pattern) and can than be reference with \g<1> or \g<name>

The following playbook shows all three variants.
For the example file

# an example
simple backref=xxxx
group backref with number=yyyy
group backref with name=zzzz

the generated output is:

# an example
P16
group backref with number=2016
group backref with name=2016

Update 2019-10-29: Replace inline code by embedded github gist to fix issue with not shown group in example.

Install inadyn-mt on CentOS7

(Update: see post "Updated inadyn-mt package for CentOS7")

I didn't found an inadyn-mt rpm package for CentOS7. So I built one myself:

• Download the source rpm for Fedora 22.
• Install the source rpm
  

  rpm -i inadyn-mt-2.24.44-1.fc22.src.rpm

• The sources are installed in the ~/rpmbuild directory.
• Move to the rpmbuild directory and run rpmbuild
  

  cd ~/rpmbuild/SPECS
  rpmbuild -ba inadyn-mt.spec
  

• That's all. The rpm can now be installed with
  

  rpm -i ~/rpmbuild/RPMS/x86_64/inadyn-mt-2.24.44-1.el7.centos.x86_64.rpm

Here you can download it: inadyn-mt-2.24.44-1.el7.centos.x86_64.rpm

ssh-add complains: Could not open a connection to your authentication agent

When running an ansible playbook a series of ssh connections is used to perform the different playbook actions. Each ssh connection requires the ssh-key passphrase to be entered. The default way to avoid this is to use ssh-agent. The ssh-agent stores your passphrase and send it to ssh if needed. To import a key into ssh-agent you use ssh-add. I try it for my ansible experiments where ansible is running in a docker container (see my previous posts)

docker run --rm -i -t -v $(pwd):/data -w /data thomo/ansible bash -c "ssh-add deploy__2015 && ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=newhosts.keys -i ./deploy__2015' ansible-playbook site.yml -i ./inventories/hosts.baremetal"
Could not open a connection to your authentication agent.

Seams that there is no ssh-agent running. The agent can be start with

eval $(ssh-agent)

But you have to take care, that the evaluation is postponed to shell in the docker - and not be done already in the host shell. So put it in single quotes:

$ ./dr 'eval $(ssh-agent) && ssh-add ./deploy__2015 && ANSIBLE_SSH_ARGS="-o UserKnownHostsFile=newhosts.keys -i ./deploy__2015" ansible-playbook site.yml -i ./inventories/hosts.baremetal'
Agent pid 8
Enter passphrase for ./deploy__2015:
Identity added: ./deploy__2015 (./deploy__2015)

PLAY [apply common configuration to all nodes] *************************
...

dr is a small script which wraps the docker run parameter

docker run --rm -i -t -w /data -v $(pwd):/data thomo/ansible bash -c "$*"

Update 2016-01-06

Improved dr script

SSHPARAM='eval $(ssh-agent) && ssh-add ./deploy__2015 && ANSIBLE_SSH_ARGS="-o UserKnownHostsFile=newhosts.keys -i ./deploy__2015"'
docker run --rm -i -t -w /data -v $(pwd):/data thomo/ansible:20160106 sh -c "${SSHPARAM} $*"

Example usage:

$ ./dr "ansible server -u deploy -m ping -i ./inventories/hosts.baremetal"
Agent pid 9
Enter passphrase for ./deploy__2015:
Identity added: ./deploy__2015 (./deploy__2015)
10.0.0.1 | success >> {
    "changed": false,
    "ping": "pong"
}

Unable to access Vagrant box from docker container running in VirtualBox

I want to setup my home server with a new OS (CentOS 7). I decided to use Ansible to provisioning the server because I want to document the different steps, and to learn Ansible. To be able to play with Ansible and to try out different steps I had the idea to use a virtual box image with CentOS7 as target (simulating my new server) and a docker container with ansible as provisioner. Both systems/boxes was setup in just a few minutes. Since I already have installed Vagrant, VirtualBox and Docker I just need to do the following

• Download and start the virtualbox
• Build a docker image with this Dockerfile by execute
  
  

  docker build -t thomo/ansible .

  
  and run it with
  

  docker run --rm -i -t -v $(pwd):/data -w /data thomo/ansible bash

Afterwards I had the shown structure

Fine - I thought at least ...

Next I tried to ping "new host" from the ansible container.

$ docker run --rm -i -t -v $(pwd):/data -w /data thomo/ansible bash
[root@07091097c2ca data]# ping 10.0.2.15
PING 10.0.2.15 (10.0.2.15) 56(84) bytes of data.
64 bytes from 10.0.2.15: icmp_seq=1 ttl=64 time=0.055 ms
64 bytes from 10.0.2.15: icmp_seq=2 ttl=64 time=0.073 ms
64 bytes from 10.0.2.15: icmp_seq=3 ttl=64 time=0.103 ms
64 bytes from 10.0.2.15: icmp_seq=4 ttl=64 time=0.108 ms
^C
--- 10.0.2.15 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.055/0.084/0.108/0.024 ms
[root@07091097c2ca data]#

Seams to work ...

Next: Login in with ssh

[root@07091097c2ca data]# ssh 10.0.2.15
The authenticity of host '10.0.2.15 (10.0.2.15)' can't be established.
ECDSA key fingerprint is d2:62:41:e4:a3:d2:40:cf:a0:02:eb:d0:16:ab:49:bc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.2.15' (ECDSA) to the list of known hosts.
root@10.0.2.15's password:
Permission denied, please try again.
root@10.0.2.15's password:
Permission denied, please try again.
root@10.0.2.15's password:
Permission denied (publickey,password,keyboard-interactive).
[root@07091097c2ca data]#

Huh, what the heck ...?

After some investigations (mainly by comparing the host key fingerprints) I realize that I did not communicate with my "new host" box but with the docker host. In fact the "new host" box and the docker host both use the VirtualBox NAT interface but the boxes can not reach each other.

In VirtualBox this router [the NAT interface] is placed between each virtual machine and the host. This separation maximizes security since by default virtual machines cannot talk to each other. (source)

You wonder why the ping trial worked? The reason is the docker host has the same ip address (10.0.2.15) as the "new host". So I did ping the docker host instead of "new host".

I hope my documentation helps other to avoid similar errors or at least facilitates troubleshooting in a similar situation.

Build your own CentOS 7 Broadcom Wireless Driver

I just finished my first experiments with Docker. To doing this I put a new SSD in my MacBook and start with my experiments. Some hours later I wanted to moving around at home and realized that the wireless network didn't work. So I found a first use case of my experiences.

To use the wireless network on the MacBook you need a Broadcom wireless driver (in my case for a BCM4322, VendorID:DeviceID 14e4:432b) which you have to build by your own as documented here. Before following the steps described there I wrote a Dockerfile which builds a Docker image with all the packages needed to build the wireless module. Also I wrote a shell script which executed the build steps to assemble the wireless module (driver).

If you need/want to build the Broadcom wireless driver for your CentOS 7 installation you just need:

• Copy the script to an empty directory, 
• download the Broadcom wireless driver source and 
• put it in the same directory
• then run the script
• continue at step 4 in the CentOS HowTo 

Hope it works.

ogg 2 mp3

Permission denied (publickey).

After successful configure github access using Win/msysgit I have to fight on the Linux front. Just copy the private key file generated with puttygen didnt work:

[thomo@sun .ssh]$ ssh git@github.com
Enter passphrase for key '/home/thomo/.ssh/thomo__github_private.ppk':
Permission denied (publickey).

I neither enter a passphrase nor was able to change the empty phrase using ssh-keygen -P. :-|

To solve the problem I export the private key in OpenSSH format:

Initd Script für cruisecontrolrb

Based on the $CCRB_HOME/daemon/cruise.sample version.

#!/usr/bin/ruby
# chkconfig: 235 99 10
# description: Start or stop the SnipSnap server
#
### BEGIN INIT INFO
# Provides: cruisecontrolrb
# Required-Start:
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: Start or stop the cruisecontrolrb daemon
### END INIT INFO

ENV['PATH']="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11"

require "fileutils"
include FileUtils

require "rubygems"

begin
  gem 'mongrel'
rescue => e
  puts "Error: daemon mode of CC.rb requires mongrel installed"
  exit 1
end

def cruise_path
  "/home/cruise/cruisecontrolrb"
end

def cruise_user
  "cruise"
end

def cruise_log
  "/var/log/cruisecontrolrb.log"
end

def start
  system "su - #{cruise_user} -c 'cd #{cruise_path} && ./cruise start -d >> #{cruise_log}'"
end

def stop
  system "mongrel_rails stop -P #{cruise_path}/tmp/pids/mongrel.pid >> #{cruise_log}" if File.exists?("#{cruise_path}/tmp/pids/mongrel.pid")
  Dir["#{cruise_path}/tmp/pids/builders/*.pid"].each do |pid_file|
    pid = File.open(pid_file){|f| f.read }
    system "kill -9 #{pid}"
    rm pid_file
  end
end

command = ARGV.shift

system "touch #{cruise_log}"
system "chown #{cruise_user} #{cruise_log}"


case command
when 'start'
  start
  exit 0
when 'stop'
  stop
  exit 0
when 'restart'
  stop
  start
  exit 0
else
  puts "Usage: /etc/init.d/cruise start|stop|restart"
  exit 1
end