If you were logged in and working on this process, you would log your work on this page.
Acquire new skills. You might want to document something if you discover something new, or update the training manuals or the documentation that the network is using.
Researched amazon machine learning.
- Studied developer's guide: http://docs.aws.amazon.com/machine-learning/latest/dg/machinelearning-dg.pdf
-Followed tutorials from the guide and from https://cloudacademy.com/
The learning lead me to write this study:
Found and read books on AWS and IoT infrastructure solutions:
Amazon Web Services in Action
Wtched recent YouTube videos about AWS and IoT
Learning the intricacies of the MQTT protocol and AWS IoT brokerage
***DOCUMENTATION COMING SOON***
Researched AWS IoT integrated services for storage, data processing, database management, notification and visualization.
Amazon Simple Storage Service—Provides scalable storage in the AWS cloud.
Amazon DynamoDB—Provides managed NoSQL databases.
Amazon Kinesis—Enables real-time processing of streaming data at a massive scale.
AWS Lambda—Runs your code on virtual servers from Amazon EC2 in response to events.
Amazon Simple Notification Service—Sends or receives notifications.
Amazon Simple Queue Service—Stores data in a queue to be retrieved by applications.
Doing outreach, seeding individual emails to people you know, setting up and attending meetings with people, all that with the goal of getting more people to work on the project.
You need to present a short description of what you did when you log.
Searching for information, designs, prototypes, doing some background checks, search prior art, ...
You need to save that search in a document and store it in the project folder (or any other location that is shared with all the project affiliates) and link to it when you log.
Researched best practices for integration of the following AWS components:
-DynamoDB , Kinesis, Machine Learning and Lambda
Researched different plugins for Freeboard
Research into problems with AWS IoT component from blogs, videos and forums.
Issue with changes discovered that BROKE rules engine.
Researched problems regarding AWS IoT and did a thorough analysis of MQTT
Researched dealing with various issues that arose during the first test run including XBee module problem with high speed serial connection, properly flushing the buffer, various data formatting ideas and others that will all be documented.
Researched AWS's IoT architecture and terminology for connecting things
Researched aws iot_beta and documented here: https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing
Set up AWS SNS (Simple Notification Service) for e-mail from IoT activity
Incorporated new load sensors into python MQTT script for pushing to AWS IoT and pushed to remote branch load_sensors for testing.
Fixed Rules Engine problem.
Setup Cloud Watch for monitoring services
Setup Thing Registry, Rules Engine , policies, certificates and IAM roles on AWS.
Tested writing to DynamoDB while monitoring with CloudWatch
Formatted script for publishing to AWS
Wrote python script for publishing to AWS using Paho MQTT for Raspberry Pi
Set up dashboard at Freeboard
Set up thing shadow and formatted receiving end points on AWS
Set up DynamoDB instance for receiving data
Tested XBee connection using previously written code:
Wrote new script to receive new data from arduino and push data to Snaplabs.
Set up a second raspberry pi using the latest raspbian "jessie" image. Encountered new problems and solved them. Everything was well documented
Spent the day with Tibi and Ahmed at Robco.
Setup the raspberry pi on Robco's intranet.
Tested programming an XBee module with a python script on the Pi.
Helped Ahmed up a simple XBee network between the Pi and an Arduino.
Helped Tibi with the logic behind the Arduino firmware.
Wrote python script to PUSH data from the raspberry pi at test site to Snaplabs.
Setup the Raspberry PI at the lab to push data to Snaplabs remote server
Fix problem with mosquitto MQTT not connecting to AWS and adnd documented problem in doc on page 8 https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing
-Setup AWS CLI on the Raspberry Pi, the command line interface for all of Amazon's Web Services including IoT
-Setup Mosquitto MQTT command line broker/client
-Wrote python script to publish temperature at the lab
-Debugged and tested script, success!
Visit http://test.mosquitto.org/gauge/ to see working demo (msg me first to publish data since this is not our broker/server)
Installed Paho-MQTT on the pi. https://eclipse.org/paho/
Wrote python script to connect to AWS. https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing page 5
Tinkered with python script to connect DS18B20 1-wire sensor to the Pi.
Configured router to expose Pi for remote SSH
Setup RasPi, aws SDK for openssl , python sdk (boto).
Mapped out AWS architecture for the next phase of the project.
Introducing Rain as software engineer and understanding how to access/use the sensing node.
Administration activities around projects.
Updated Redmine requirements and checklist for Cloud Analytics
Stigmergy is part of our functioning, meaning leaving as many traces of your work as possible, including instructions for those who might want to build on our work.
You must document in a place that everyone agrees and can find. The documentation must be open to All affiliates to edit.
Discussion with Pov about a better way for timeline diagrams. That lead to the integrated timeline drawing:
Timelines drawings for all the sensors:
Timeline for tachometer
Flow rate sensor
Fluid level and Solenoid
Propose different options for drawings showing the timeline for sensor readings (according to discussion with Pov). Starting with the position and load sensors:
Flowchart for Shaft temperature
OneWire temperature & FFT
Understanding the processes for tachometer and flow rate. Drawing of the flowchart for those processes https://docs.google.com/drawings/d/1vWCUaplCoJBYliafV_lq6VIf6-DlGhtGlZ8QV1Rffxs
Understanding the processes for fluid level and solenoid control. Drawing of the flowchart for those processes https://docs.google.com/drawings/d/17740kh8EILxwCytr4SiByd_8bFEphFYWeLNmDrAmeuc
Discussing with Pov about the best format for flowcharts.
Understanding the process for position sensor.Drawing of the flowchart for that process and the smoothing (running average) function https://docs.google.com/drawings/d/19Fi3GyeFTa6VpcuPmV1fsZD3EZIJkrA_WeNN30wiT_E
Understanding the Arduino firmware at high level. Drawing of the flowchart of the general firmware process.
Integration of the flowchart in the main document by creating the new section "Arduino implementation".
Writing Amazon Machine Learning analysis document
Fully documented base station (raspberry pi) set up with pictures and step by step instructions in the document.
Added Mosquitto commands to Github
Documented in the doc: https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing
-Installation of the AWS CLI tools
-Setting up the tools for IoT
-Setting up and configuring a thing in the registry using the CLI
-Setting up and configuring a thing using the web console
Full instructions with images
Documented work done with MQTT in page 7 of https://docs.google.com/document/d/1s3-n9nbJ76C_NYesNoiGCFTrnO7Qua-jF9Pkpd45Ba0/edit?usp=sharing
Software solution architecture document presented to Robco.
Participating in a gathering to discuss, plan, analyse, etc. Might require some traveling. Meetings can also be virtual. Might require documentation and further communication.
Phone conversation with Nick Flaquer
email@example.com | (206) 922-5272
Certified Solutions Architect
Can call him anytime for help :)
Presented Amazon Machine Learning at Robco to John, Lai, Pat and Jean.
Scrum meeting at Robco
Conference call scrum meeting
Conference call with the paying organization to discuss software choice, and the platform we can offer.
Facilitation means guiding affiliates, helping them with technical issues, navigation issues, administrative issues, ... helping
Worked with Mourad and partner Rain, who came to the lab to work on sending data over the Interent, We coordinated on a Raspberry Pi + sensor connection
Context: Sensor network
Order: Work order 187 , Sensor network Cycle 1 Cloud analytics due: 2015-11-24