Freebox Revolution Monitoring - Initial commit

README.md

@@ -1,2 +1,148 @@
 # freebox-revolution-monitoring
 Simple Freebox Revolution Monitoring with SexiGraf 
+
+![freebox monitoring dashboard](freebox_dashboard.png)
+
+Based/Inspired by https://github.com/tuxtof/freebox-monitoring
+
+The newer freebox devices don't offer the hosted file with all the data information usually accessible (`http://mafreebox.freebox.fr/pub/fbx_info.txt`)
+
+So this solution is leveraging the [Freebox API](http://dev.freebox.fr/sdk/os/) but just focuses on the stats I'm interested in, if you want to extend the script check all the other things available [here](http://dev.freebox.fr/sdk/os/connection/)
+
+# Pre-requisites
+
+This is what I used, you can of course adapt the collector script to talk to influxdb or whatever :-)
+
+- [SexiGraf](http://www.sexigraf.fr) or any Grafana/Graphite stack
+- [Telegraf](https://github.com/influxdata/telegraf)
+- Python with `json` & `requests` libraries installed
+- Physical Access to the Freebox Server device
+
+# Step 1: Register an app with the Freebox device
+
+First thing to do is to register an app, to generate a specific App Token.
+
+Use the `freebox_register_app.py` script.
+
+*PS: You can modify the app name/versions etc as shown below (Optional)*
+
+```python
+    app_info = {
+        'app_id': 'fr.freebox.seximonitor',
+        'app_name': 'SexiMonitor',
+        'app_version': '0.4.2',
+        'device_name': 'SexiServer'
+    }
+``` 
+
+Once you execute this script, you will see something similar to this:
+
+![register](freebox_registration.png)
+
+Head to your Freebox Derver device.
+
+![Freebox Server Validation](seximonitor_register.jpg)
+
+Press the `>` to authorize the app registration process.
+
+Be sure to save the token somewhere safe, you will need it to authenticate against the freebox api afterwards :)
+
+# Step 2: Use the script to display freebox statistics information
+
+Once you have your App Token, the process to authenticate happens in 2 steps:
+- Fetch the current `challenge` (basically a random generated string changing over time)
+- Compute a `session password` with the `challenge` and your `App Token`
+
+(This avoids sending the token over the network)
+
+Edit the freebox_monitor.py script and set your `App token/Track ID` (line 73-74)
+
+```python
+    freebox_app_token = "CHANGE_THIS"
+    track_id = "CHANGE_THIS"
+```
+
+then execute it, to make sure it connects and display information
+
+![freebox monitor](freebox_monitor.png)
+
+# Step 3: Leverage telegraf to call the script and send it to Graphite
+
+Install telegraf on the SexiGraf appliance.
+
+```console
+wget https://dl.influxdata.com/telegraf/releases/telegraf_1.0.1_amd64.deb
+dpkg -i telegraf_1.0.1_amd64.deb 
+```
+
+Generate a config file for our plugins `exec` and `graphite`
+
+```console
+telegraf --input-filter exec --output-filter graphite config > /etc/telegraf/telegraf.conf
+```
+
+You can then check & edit the configuration file to make it look as follows:
+
+```ini
+###############################################################################
+#                            OUTPUT PLUGINS                                   #
+###############################################################################
+
+# Configuration for Graphite server to send metrics to
+[[outputs.graphite]]
+  ## TCP endpoint for your graphite instance.
+  ## If multiple endpoints are configured, output will be load balanced.
+  ## Only one of the endpoints will be written to with each iteration.
+  servers = ["localhost:2003"]
+  ## Prefix metrics name
+  prefix = ""
+  ## Graphite output template
+  ## see https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
+  template = "host.tags.measurement.field"
+  ## timeout in seconds for the write connection to graphite
+  timeout = 2
+
+###############################################################################
+#                            INPUT PLUGINS                                    #
+###############################################################################
+
+# Read metrics from one or more commands that can output to stdout
+[[inputs.exec]]
+  ## Commands array
+  command = "/usr/local/freebox-revolution-monitoring/freebox_monitor.py"
+
+  ## Timeout for each command to complete.
+  timeout = "5s"
+
+  ## Data format to consume.
+  ## Each data format has it's own unique set of configuration options, read
+  ## more about them here:
+  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
+  data_format = "graphite"
+```
+
+Be sure to copy your modified `freebox_monitor.py` script to `/usr/local/freebox-revolution-monitoring/`
+
+Relaunch telegraf and check the logs
+
+```console
+root@sexigraf:~# tail -f /var/log/telegraf/telegraf.log
+2016/12/11 18:26:30 Output [graphite] buffer fullness: 7 / 10000 metrics. Total gathered metrics: 675367. Total dropped metrics: 0.
+2016/12/11 18:26:30 Output [graphite] wrote batch of 7 metrics in 165.892µs
+2016/12/11 18:26:40 Output [graphite] buffer fullness: 7 / 10000 metrics. Total gathered metrics: 675374. Total dropped metrics: 0.
+2016/12/11 18:26:40 Output [graphite] wrote batch of 7 metrics in 169.849µs
+2016/12/11 18:26:50 Output [graphite] buffer fullness: 7 / 10000 metrics. Total gathered metrics: 675381. Total dropped metrics: 0.
+2016/12/11 18:26:50 Output [graphite] wrote batch of 7 metrics in 183.453µs
+2016/12/11 18:27:00 Output [graphite] buffer fullness: 7 / 10000 metrics. Total gathered metrics: 675388. Total dropped metrics: 0.
+2016/12/11 18:27:00 Output [graphite] wrote batch of 7 metrics in 156.956µs
+2016/12/11 18:27:10 Output [graphite] buffer fullness: 7 / 10000 metrics. Total gathered metrics: 675395. Total dropped metrics: 0.
+2016/12/11 18:27:10 Output [graphite] wrote batch of 7 metrics in 170.216µs
+2016/12/11 18:27:20 Output [graphite] buffer fullness: 7 / 10000 metrics. Total gathered metrics: 675402. Total dropped metrics: 0.
+2016/12/11 18:27:20 Output [graphite] wrote batch of 7 metrics in 177.338µs
+```
+
+If the output is similar to this, you should be good to go and build your dashboards in SexiGraf.
+Here is a 2 day view of the download/upload stats.
+
+![dashboard 2days](freebox_2days.png)
+

freebox_monitor.py

@@ -0,0 +1,124 @@
+#!/usr/bin/env python
+# pylint: disable=C0103,C0111,W0621
+
+
+#
+# Freebox API SDK / Docs: http://dev.freebox.fr/sdk/os/login/
+#
+
+import os
+import json
+import hmac
+import time
+
+from hashlib import sha1
+import requests
+
+
+def get_challenge(freebox_app_id):
+    api_url = 'http://mafreebox.freebox.fr/api/v3/login/authorize/%s' % freebox_app_id
+
+    r = requests.get(api_url)
+
+    if r.status_code == 200:
+        return r.json()
+    else:
+        print 'Failed request: %s\n' % r.text
+
+def open_session(password, freebox_app_id):
+    api_url = 'http://mafreebox.freebox.fr/api/v3/login/session/'
+
+    app_info = {
+        'app_id': freebox_app_id,
+        'password': password
+    }
+    json_payload = json.dumps(app_info)
+
+    r = requests.post(api_url, data=json_payload)
+
+    if r.status_code == 200:
+        return r.json()
+    else:
+        print 'Failed request: %s\n' % r.text
+
+
+
+def get_connection_stats(headers):
+    api_url = 'http://mafreebox.freebox.fr/api/v3/connection/'
+
+    r = requests.get(api_url, headers=headers)
+
+    if r.status_code == 200:
+        return r.json()
+    else:
+        print 'Failed request: %s\n' % r.text
+
+
+
+def get_ftth_status(headers):
+    api_url = 'http://mafreebox.freebox.fr/api/v3/connection/ftth/'
+
+    r = requests.get(api_url, headers=headers)
+
+    if r.status_code == 200:
+        return r.json()
+    else:
+        print 'Failed request: %s\n' % r.text
+
+
+# Main
+if __name__ == '__main__':
+
+    freebox_app_id = "fr.freebox.seximonitor"
+    freebox_app_token = "CHANGE_THIS"
+    track_id = "CHANGE_THIS"
+
+    # Fetch challenge
+    resp = get_challenge(track_id)
+    challenge = resp['result']['challenge']
+
+    # Generate session password
+    h = hmac.new(freebox_app_token, challenge, sha1)
+    password = h.hexdigest()
+
+    # Fetch session_token
+    resp = open_session(password, freebox_app_id)
+    session_token = resp['result']['session_token']
+
+    # Setup headers with the generated session_token
+    headers = {
+        'X-Fbx-App-Auth': session_token
+    }
+
+    # Setup hashtable for results
+    myData = {}
+
+    # Fetch connection stats
+    jsonRaw = get_connection_stats(headers)
+
+    myData['bytes_down'] = jsonRaw['result']['bytes_down']
+    myData['bytes_up'] = jsonRaw['result']['bytes_up']
+    myData['rate_down'] = jsonRaw['result']['rate_down']
+    myData['rate_up'] = jsonRaw['result']['rate_up']
+    if jsonRaw['result']['state'] == "up":
+        myData['state'] = 1
+    else:
+        myData['state'] = 0
+
+    # Fetch ftth signal stats
+    jsonRaw = get_ftth_status(headers)
+
+    myData['sfp_pwr_rx'] = jsonRaw['result']['sfp_pwr_rx']
+    myData['sfp_pwr_tx'] = jsonRaw['result']['sfp_pwr_tx']
+
+    # Prepping Graphite Data format
+    timestamp = int(time.time())
+
+    # Output the information
+    print "freebox.bytes_down %s %d" % (myData['bytes_down'], timestamp)
+    print "freebox.bytes_up %s %d" % (myData['bytes_up'], timestamp)
+    print "freebox.rate_down %s %d" % (myData['rate_down'], timestamp)
+    print "freebox.rate_up %s %d" % (myData['rate_up'], timestamp)
+    print "freebox.state %s %d" % (myData['state'], timestamp)
+    print "freebox.sfp_pwr_rx %s %d" % (myData['sfp_pwr_rx'], timestamp)
+    print "freebox.sfp_pwr_tx %s %d" % (myData['sfp_pwr_tx'], timestamp)

freebox_register_app.py

@@ -0,0 +1,30 @@
+#!/usr/bin/env python
+# pylint: disable=C0103,C0111
+
+import json
+import requests
+
+def get_authorization():
+    api_url = 'http://mafreebox.freebox.fr/api/v3/login/authorize/'
+    headers = {'Content-type': 'application/json'}
+    app_info = {
+        'app_id': 'fr.freebox.seximonitor',
+        'app_name': 'SexiMonitor',
+        'app_version': '0.4.2',
+        'device_name': 'SexiServer'
+    }
+    json_payload = json.dumps(app_info)
+
+    r = requests.post(api_url, headers=headers, data=json_payload)
+
+    if r.status_code == 200:
+        return r.json()
+    else:
+        print 'Failed registration: %s\n' % r.text
+
+if __name__ == '__main__':
+    resp = get_authorization()
+
+    print '[Track ID] {}'.format(resp['result']['track_id'])
+    print '[App token] {}'.format(resp['result']['app_token'])
+    print 'Press on the right arrow on the Freebox Server and validate the app registration'

telegraf.conf

@@ -0,0 +1,106 @@
+# Telegraf Configuration
+#
+# Telegraf is entirely plugin driven. All metrics are gathered from the
+# declared inputs, and sent to the declared outputs.
+#
+# Plugins must be declared in here to be active.
+# To deactivate a plugin, comment out the name and any variables.
+#
+# Use 'telegraf -config telegraf.conf -test' to see what metrics a config
+# file would generate.
+#
+# Environment variables can be used anywhere in this config file, simply prepend
+# them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
+# for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)
+
+
+# Global tags can be specified here in key="value" format.
+[global_tags]
+  # dc = "us-east-1" # will tag all metrics with dc=us-east-1
+  # rack = "1a"
+  ## Environment variables can be used as tags, and throughout the config file
+  # user = "$USER"
+
+
+# Configuration for telegraf agent
+[agent]
+  ## Default data collection interval for all inputs
+  interval = "10s"
+  ## Rounds collection interval to 'interval'
+  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
+  round_interval = true
+
+  ## Telegraf will send metrics to outputs in batches of at
+  ## most metric_batch_size metrics.
+  metric_batch_size = 1000
+  ## For failed writes, telegraf will cache metric_buffer_limit metrics for each
+  ## output, and will flush this buffer on a successful write. Oldest metrics
+  ## are dropped first when this buffer fills.
+  metric_buffer_limit = 10000
+
+  ## Collection jitter is used to jitter the collection by a random amount.
+  ## Each plugin will sleep for a random time within jitter before collecting.
+  ## This can be used to avoid many plugins querying things like sysfs at the
+  ## same time, which can have a measurable effect on the system.
+  collection_jitter = "0s"
+
+  ## Default flushing interval for all outputs. You shouldn't set this below
+  ## interval. Maximum flush_interval will be flush_interval + flush_jitter
+  flush_interval = "10s"
+  ## Jitter the flush interval by a random amount. This is primarily to avoid
+  ## large write spikes for users running a large number of telegraf instances.
+  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
+  flush_jitter = "0s"
+
+  ## By default, precision will be set to the same timestamp order as the
+  ## collection interval, with the maximum being 1s.
+  ## Precision will NOT be used for service inputs, such as logparser and statsd.
+  ## Valid values are "ns", "us" (or "µs"), "ms", "s".
+  precision = ""
+  ## Run telegraf in debug mode
+  debug = false
+  ## Run telegraf in quiet mode
+  quiet = false
+  ## Override default hostname, if empty use os.Hostname()
+  hostname = ""
+  ## If set to true, do no set the "host" tag in the telegraf agent.
+  omit_hostname = false
+
+
+###############################################################################
+#                            OUTPUT PLUGINS                                   #
+###############################################################################
+
+# Configuration for Graphite server to send metrics to
+[[outputs.graphite]]
+  ## TCP endpoint for your graphite instance.
+  ## If multiple endpoints are configured, output will be load balanced.
+  ## Only one of the endpoints will be written to with each iteration.
+  servers = ["localhost:2003"]
+  ## Prefix metrics name
+  prefix = ""
+  ## Graphite output template
+  ## see https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
+  template = "host.tags.measurement.field"
+  ## timeout in seconds for the write connection to graphite
+  timeout = 2
+
+
+
+###############################################################################
+#                            INPUT PLUGINS                                    #
+###############################################################################
+
+# Read metrics from one or more commands that can output to stdout
+[[inputs.exec]]
+  ## Commands array
+  command = "/usr/local/freebox-revolution-monitoring/freebox_monitor.py"
+
+  ## Timeout for each command to complete.
+  timeout = "5s"
+
+  ## Data format to consume.
+  ## Each data format has it's own unique set of configuration options, read
+  ## more about them here:
+  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
+  data_format = "graphite"