Process: Sensor network Cycle 1 Cloud analytics starting 2015-11-24 ending 2015-11-24

If you were logged in and working on this process, you would log your work on this page.

Outputs

Scheduled:

Prototype Software - computer program 1.00 Time - Hours due Nov. 24, 2015

Work

Planned Work: (Requirements are ordered by due date)

Work - Training self: 1.00 Time - Hours due Nov. 24, 2015

Acquire new skills. You might want to document something if you discover something new, or update the training manuals or the documentation that the network is using.

Unassigned
Work events:
Feb. 17, 2016 8.00 Time - Hours Done by Jim

Researched amazon machine learning.
- Studied developer's guide: http://docs.aws.amazon.com/machine-learning/latest/dg/machinelearning-dg.pdf

-Followed tutorials from the guide and from https://cloudacademy.com/

The learning lead me to write this study:
https://drive.google.com/open?id=1ASXpTxCDmBoGYFlQnWxpwarfdxv1mljT9IwNfR0NNFE

-

Feb. 11, 2016 6.00 Time - Hours Done by Jim

Found and read books on AWS and IoT infrastructure solutions:
Enterprise IoT
https://drive.google.com/open?id=0B-s1IR1iOMsNWTdrdjRwWXA3VnM
Amazon Web Services in Action
https://drive.google.com/open?id=0B-s1IR1iOMsNSm5wOElKV01sTlk

Feb. 10, 2016 2.00 Time - Hours Done by Jim

Wtched recent YouTube videos about AWS and IoT
https://www.youtube.com/results?q=aws+iot&sp=CAISAA%253D%253D

Jan. 20, 2016 3.00 Time - Hours Done by Jim

Learning the intricacies of the MQTT protocol and AWS IoT brokerage
***DOCUMENTATION COMING SOON***

Dec. 8, 2015 4.00 Time - Hours Done by Jim

Researched AWS IoT integrated services for storage, data processing, database management, notification and visualization.

Amazon Simple Storage Service—Provides scalable storage in the AWS cloud.
http://aws.amazon.com/s3/
Amazon DynamoDB—Provides managed NoSQL databases.
http://aws.amazon.com/dynamodb/
Amazon Kinesis—Enables real-time processing of streaming data at a massive scale.
http://aws.amazon.com/kinesis/
AWS Lambda—Runs your code on virtual servers from Amazon EC2 in response to events.
http://aws.amazon.com/lambda/
Amazon Simple Notification Service—Sends or receives notifications.
http://aws.amazon.com/sns/
Amazon Simple Queue Service—Stores data in a queue to be retrieved by applications.
http://aws.amazon.com/sqs/

Work - Recruiting: 1.00 Each due Nov. 24, 2015

Doing outreach, seeding individual emails to people you know, setting up and attending meetings with people, all that with the goal of getting more people to work on the project.
You need to present a short description of what you did when you log.

Unassigned
Work - Information mining: 1.00 Time - Hours due Nov. 24, 2015

Searching for information, designs, prototypes, doing some background checks, search prior art, ...
You need to save that search in a document and store it in the project folder (or any other location that is shared with all the project affiliates) and link to it when you log.

Unassigned
Work events:
Feb. 12, 2016 8.00 Time - Hours Done by Jim

Researched best practices for integration of the following AWS components:
-DynamoDB , Kinesis, Machine Learning and Lambda
Researched different plugins for Freeboard
https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing

Feb. 2, 2016 2.00 Time - Hours Done by Jim

Research into problems with AWS IoT component from blogs, videos and forums.
Issue with changes discovered that BROKE rules engine.

Jan. 15, 2016 2.00 Time - Hours Done by Jim

Researched problems regarding AWS IoT and did a thorough analysis of MQTT

Jan. 12, 2016 2.00 Time - Hours Done by Jim

Researched dealing with various issues that arose during the first test run including XBee module problem with high speed serial connection, properly flushing the buffer, various data formatting ideas and others that will all be documented.

Dec. 3, 2015 1.00 Time - Hours Done by Jim

Researched AWS's IoT architecture and terminology for connecting things

Nov. 28, 2015 2.00 Time - Hours Done by Jim
Work - Programming for product: 1.00 Time - Hours due Nov. 24, 2015
Unassigned
Work events:
Feb. 16, 2016 3.00 Time - Hours Done by Jim

Set up Flask python web server on RPi serving client side dashboard Freeboard.
https://github.com/mitsuhiko/flask
https://github.com/Freeboard/freeboard
For now the dashboard is displaying the RPi's performance (cpu usage, memory load, etc) and on off activity.

Feb. 16, 2016 1.00 Time - Hours Done by Jim

Set up AWS SNS (Simple Notification Service) for e-mail from IoT activity

Feb. 16, 2016 0.25 Time - Hours Done by Jim

Incorporated new load sensors into python MQTT script for pushing to AWS IoT and pushed to remote branch load_sensors for testing.
https://github.com/Sensorica/Sensor-Network/blob/load_sensor/Raspberry_Pi/paho_test.py

Feb. 3, 2016 3.00 Time - Hours Done by Jim

Fixed Rules Engine problem.
Created DynamoDB
Tested Kinesis
Setup Cloud Watch for monitoring services

Jan. 28, 2016 3.00 Time - Hours Done by Jim

Setup Thing Registry, Rules Engine , policies, certificates and IAM roles on AWS.
Tested writing to DynamoDB while monitoring with CloudWatch

Jan. 15, 2016 6.00 Time - Hours Done by Jim

Formatted script for publishing to AWS
https://github.com/Sensorica/Sensor-Network/blob/master/Raspberry_Pi/Mosquitto_aws_publish_test.txt
Wrote python script for publishing to AWS using Paho MQTT for Raspberry Pi
https://github.com/Sensorica/Sensor-Network/blob/master/Raspberry_Pi/paho_test.py
Set up dashboard at Freeboard
https://freeboard.io/board/RQFab9
Set up thing shadow and formatted receiving end points on AWS
Set up DynamoDB instance for receiving data

Jan. 12, 2016 2.50 Time - Hours Done by Jim

Tested XBee connection using previously written code:
https://github.com/Sensorica/Sensor-Network/blob/master/Raspberry_Pi/XBee_AT_test.py
Wrote new script to receive new data from arduino and push data to Snaplabs.
https://github.com/Sensorica/Sensor-Network/blob/master/Raspberry_Pi/testbench_to_snaplabs.py

Dec. 17, 2015 3.00 Time - Hours Done by Jim

Set up a second raspberry pi using the latest raspbian "jessie" image. Encountered new problems and solved them. Everything was well documented

Dec. 15, 2015 4.00 Time - Hours Done by Jim

Spent the day with Tibi and Ahmed at Robco.
Setup the raspberry pi on Robco's intranet.
Tested programming an XBee module with a python script on the Pi.
Helped Ahmed up a simple XBee network between the Pi and an Arduino.
Helped Tibi with the logic behind the Arduino firmware.

Dec. 14, 2015 0.50 Time - Hours Done by Jim

Wrote python script to PUSH data from the raspberry pi at test site to Snaplabs.

Dec. 11, 2015 0.50 Time - Hours Done by Jim

Setup the Raspberry PI at the lab to push data to Snaplabs remote server

Dec. 7, 2015 1.00 Time - Hours Done by Jim

Fix problem with mosquitto MQTT not connecting to AWS and adnd documented problem in doc on page 8 https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing

Dec. 4, 2015 4.00 Time - Hours Done by Jim

-Setup AWS CLI on the Raspberry Pi, the command line interface for all of Amazon's Web Services including IoT
-Setup Mosquitto MQTT command line broker/client
-Wrote python script to publish temperature at the lab
-Debugged and tested script, success!
Visit http://test.mosquitto.org/gauge/ to see working demo (msg me first to publish data since this is not our broker/server)

Dec. 3, 2015 3.00 Time - Hours Done by Jim

Installed Paho-MQTT on the pi. https://eclipse.org/paho/
Wrote python script to connect to AWS. https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing page 5
Tinkered with python script to connect DS18B20 1-wire sensor to the Pi.
Configured router to expose Pi for remote SSH

Nov. 28, 2015 3.00 Time - Hours Done by Jim

Setup RasPi, aws SDK for openssl , python sdk (boto).

Work - Design software: 1.00 Time - Hours due Nov. 24, 2015
Unassigned
Work events:
Feb. 9, 2016 2.00 Time - Hours Done by Jim

Mapped out AWS architecture for the next phase of the project.
https://drive.google.com/open?id=0B-s1IR1iOMsNZVVVaGtVM1RpXzg

Dec. 2, 2015 4.00 Time - Hours Done by mouradchtioui

Introducing Rain as software engineer and understanding how to access/use the sensing node.

Work - Administration: 1.00 Time - Hours due Nov. 24, 2015

Administration activities around projects.

Unassigned
Work events:
Feb. 11, 2016 0.50 Time - Hours Done by Jim

Updated Redmine requirements and checklist for Cloud Analytics

Work - Documenting: 1.00 Time - Hours due Nov. 24, 2015

Stigmergy is part of our functioning, meaning leaving as many traces of your work as possible, including instructions for those who might want to build on our work.
You must document in a place that everyone agrees and can find. The documentation must be open to All affiliates to edit.

Unassigned
Work events:
Nov. 3, 2016 1.50 Time - Hours Done by Humberto

Discussion with Pov about a better way for timeline diagrams. That lead to the integrated timeline drawing:
https://docs.google.com/drawings/d/1Q4XwC87hf_E71_q3KlcGOhTNBVFKCvqYpX4L_p1GHa4

Oct. 28, 2016 1.50 Time - Hours Done by Humberto

Propose different options for drawings showing the timeline for sensor readings (according to discussion with Pov). Starting with the position and load sensors:

https://docs.google.com/drawings/d/16RF7z7KA5Aus2Dlb-T5el3Q4tn17_T1Apynym1cx64Q

Oct. 14, 2016 1.50 Time - Hours Done by Humberto

Understanding the processes for tachometer and flow rate. Drawing of the flowchart for those processes https://docs.google.com/drawings/d/1vWCUaplCoJBYliafV_lq6VIf6-DlGhtGlZ8QV1Rffxs

Oct. 13, 2016 2.00 Time - Hours Done by Humberto

Understanding the processes for fluid level and solenoid control. Drawing of the flowchart for those processes https://docs.google.com/drawings/d/17740kh8EILxwCytr4SiByd_8bFEphFYWeLNmDrAmeuc
Discussing with Pov about the best format for flowcharts.

Oct. 12, 2016 1.50 Time - Hours Done by Humberto

Understanding the process for position sensor.Drawing of the flowchart for that process and the smoothing (running average) function https://docs.google.com/drawings/d/19Fi3GyeFTa6VpcuPmV1fsZD3EZIJkrA_WeNN30wiT_E

Oct. 11, 2016 2.00 Time - Hours Done by Humberto

Understanding the Arduino firmware at high level. Drawing of the flowchart of the general firmware process.
https://docs.google.com/drawings/d/1WPFtDoCInX2j54p1fKB-H6_tvjPLT9KdirrIScCMCA0

Integration of the flowchart in the main document by creating the new section "Arduino implementation".

Feb. 20, 2016 2.00 Time - Hours Done by Jim
Dec. 17, 2015 2.00 Time - Hours Done by Jim

Fully documented base station (raspberry pi) set up with pictures and step by step instructions in the document.

Dec. 16, 2015 0.25 Time - Hours Done by Jim
Dec. 8, 2015 5.00 Time - Hours Done by Jim

Documented in the doc: https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing
-Installation of the AWS CLI tools
-Setting up the tools for IoT
-Setting up and configuring a thing in the registry using the CLI
-Setting up and configuring a thing using the web console
Full instructions with images

Dec. 4, 2015 1.00 Time - Hours Done by Jim
Dec. 3, 2015 1.50 Time - Hours Done by mouradchtioui

Software solution architecture document presented to Robco.

Work - Meeting: 1.00 Time - Hours due Nov. 24, 2015

Participating in a gathering to discuss, plan, analyse, etc. Might require some traveling. Meetings can also be virtual. Might require documentation and further communication.

Unassigned
Work events:
March 3, 2016 0.50 Time - Hours Done by Jim

Phone conversation with Nick Flaquer
flaquer@amazon.com | (206) 922-5272
Certified Solutions Architect
Can call him anytime for help :)

Feb. 29, 2016 2.50 Time - Hours Done by Jim

Presented Amazon Machine Learning at Robco to John, Lai, Pat and Jean.

Dec. 15, 2015 0.50 Time - Hours Done by Jim

Scrum meeting at Robco

Dec. 10, 2015 0.50 Time - Hours Done by Jim

Conference call scrum meeting

Dec. 3, 2015 0.50 Time - Hours Done by mouradchtioui

Conference call with the paying organization to discuss software choice, and the platform we can offer.

Work - Facilitation and coordination: 1.00 Time - Hours due Nov. 24, 2015

Facilitation means guiding affiliates, helping them with technical issues, navigation issues, administrative issues, ... helping

Unassigned
Work events:
Dec. 2, 2015 1.00 Time - Hours Done by Jim

Worked with Mourad and partner Rain, who came to the lab to work on sending data over the Interent, We coordinated on a Raspberry Pi + sensor connection

Process context:

Pattern: Software Development
Context: Sensor network
Order: Work order 187 , Sensor network Cycle 1 Cloud analytics due: 2015-11-24

Previous processes:

Next processes: