My name is Hunter Lien and I am one of the new additions to the OPEnSampler team! I’m currently a senior here at Oregon State University getting my degree in computer science focusing on computer security. I first got into computer science back in freshman year of highschool when my school started a computer science program. Every level of the class was taught by a single teacher, Mr. Bartlo and I consider him the biggest contributing factor to my interest in this field. He always encouraged us to move outside our comfort zone and even helped us get funding for projects that might have required additional hardware.
Travis, Chase, and I make up the senior capstone group responsible for developing an Android application capable of interfacing with the OPEnSampler hardware. We will be working on this for the next 6 months at which point we aim to have a functioning product. All three of us will be posting weekly blog posts on the OPEnSampler website to let you all know how progress is going in whatever area of the application we happen to be working on. I’m really excited to get started with this project and can’t wait to be part of it’s success.
Earlier this year, OPEnS Lab submitted several project proposals to Oregon State University’s capstone course for computer science (CS) students. Capstone (or Senior Design) is a three-term course in which students work in small teams on projects that solve real-world problems. One of OPEnS Lab’s proposals was to enhance the OPEnSampler with GSM support enabling long-distance status updates, and to ease the sampler’s configuration with a mobile app that will communicate with the Sampler over Bluetooth. That’s where we come in!
(If seeing PDFs in git hurts you, rest assured we’ll be doing some cleanup and reorganization in the near future.)
Right now, we’re mostly getting started by preparing written documents that will guide our future work. The week before last we finalized our problem statement, and this week we’ve been working on drafting a requirements specification. Although we don’t have an exact time-line laid out, our end game is to have completed this project by OSU’s Undergrad Engineering Expo during the Spring (where we will be presenting our contributions).
A Bit About Me:
I’m Travis Whitehead, a Computer Science student at OSU with the exciting opportunity to work on the OPEnSampler for my capstone project (along with my teammates Hunter and Chase– who will also be introducing themselves in separate posts). As I don’t have a lot of background experience with mobile development, microcontrollers, GSM, or Bluetooth specifically– I’m expecting to learn a lot this year!
As a free software enthusiast, I’m delighted about the “Openly Published” aspect to OPEnS Lab. In my spare time, I work as a Student Systems Engineer at OSU’s Open Source Lab a (similar sounding) organization that provides various forms of hosting for open-source projects. Luckily, there’s room in my heart for more than one open lab.
This is the first of many updates I’ll be writing as we continue to work on this project– So stay tuned!
Hello, my name is Chase Coltman, I one of the three capstone students working on the new companion app for the OPEns Lab Water Sampler, OpenSampler. I am excited to bring some of my previous mobile development experience to this project and make a great addition to this team. Currently, I am in my Senior year at Oregon State University, and I am studying Applied Computer Science, with a focus in Simulation and Game Programming. I am thrilled to be a part of this team and can’t wait to see what’s in store.
Previously here at Oregon State, I have taken several classes that will be very beneficial to our assignment. Two of the most beneficial classes I feel will contribute the most to this project are Mobile Software and Cloud Development, and Intro to Usability Engineering. My Mobile Software and Cloud Development class will likely be the most useful as we focused on app development, however we did not cover certain things like GSM or BLE, which is going to play a very large role in the companion app. My other class, Intro to Usability Engineering, was more about general UI design, good user/design focused elements such as Affordance, Consistency and of course Usability.
One problem that I’ve been having with burying the Smartrac Dogbone tags is that soil is that they break after some use. After taking a close look at some of the broken tags it appears that this is caused by the abrasive soil eventually cutting through the outer lamination of the Dogbone tags and then damaging the IC that controls the tag, causing them to become unresponsive. To counter this, I designed a two part protective enclosure for the tag that will hopefully protect them from damage. The design is outlined below:
This “Dogbone Sandwich Shield” is made up of two identical 3-D printed parts. The sandwich ends were designed to be slightly larger than the Dogbone’s footprint with two tabs on the long ends. The tabs were placed like this to minimize interference with the moisture sensitive area of the tag which sits at its center.
Assembling the product was a fairly simple process. First, the Dogbone tag was stickered onto one of the sandwich ends. Then a small amount of acetone was brushed onto all of the sandwich end’s tabs. They were then pressed together to create a sandwich with the Dogbone in the middle. The sandwich was then treated in an acetone air bath to seal the edges. After 20 seconds in the air bath, the part was removed and put in a vice under a small amount of even pressure to make it even until it hardened. It takes around a day to fully harden, but it is tough enough after an hour. Below is a picture of the final shield.
Results
The presence of the shield surrounding the Dogbone tag made it slightly less sensitive to moisture, however, the variance from readings with the shield was no greater than it was without them. Therefore, using the shield will not influence the accuracy of the sensor once, but it will require some re-calibration.
Thinking Ahead
Thinking ahead, creating shields like this one with varying thickness could allow for changing the sensitivity of the tags. Currently, without a shield, the tags read their lowest value before the soil has gotten completely saturated and are able to read soil that is more than completely dry. Adding a shield that is a certain depth should shift the reading range to be centered and allow the tags to read over a larger range of soil conditions than they currently do.
Over the last two months, our OPEnS Data logging hub in the HJ Andrews forest has relayed over 100,000 data points to our Google Spreadsheet through the transmission protocol below. This post will provide a brief look at some interesting trends found within our dataset.
Data Transmission Diagram
Objective
The Objective of this post is to look at some POI within our data set including some interesting plots of data during the Great North American Eclipse.
Data
After collecting data points every 5 minutes for a little over two months we were fortunate to begin looking at the wealth of valuable data we had collected. As I began to analyze specific areas of the data that I found interesting, I was able to see some pretty fascinating trends that one would expect to see such as the Inverse relationship between relative humidity and temperature. I then began exploring other trends within the data and found that although some trends were expected some were not.
Graph showing the inverse relationship of relative humidity and temperature
It became clear after looking at the load cell data (well after all water we added to the system would have evaporated) that there was a periodic fluctuation much greater than any residual noise would account for. When plotted alongside daily temperature, it was clear that fluctuations were temperature dependant and we deducted that the exposed aluminum force gauge was expanding with high afternoon temperatures and contracting with colder overnight lows. This behavior visible in the graph below.
Our data is not without its issues, We found our load cell (evaporation mass) data was greatly compromised by the thermal expansion properties of the aluminum rod throughout daily temperature fluctuations.
All issues aside, we were able to capture some pretty remarkable data during the Great North American Eclipse on August 21st, the below plot focuses on the point of totality, and how this event altered the standard linear trend of light intensity increase throughout a morning with a very steep fallout in both the Infared and Full spectrum light intensity. This anomely can be viewed below.
Light Intensity Data during the Total Eclipse 8/21
Conclusions
After briefly looking at the data from this spreadsheet it clear that a more in-depth analysis will be required to really gain an understanding of the relationships and trends within the data set. However, even at a glancing look, it is clear from the strain gauge and temperature relationship will be a major consideration in the redesign and further models of our sensor suites.
As our projects continue to evolve alongside with the advances of technology it is important to keep the OPEnS lab up to date with the latest and greatest prototyping technologies at our disposal. There is no example greater than that of the microprocessor. This post will explore two of the most advanced, widely available microprocessors on the market today.
Background and Objective
Since 2010 the Arduino Uno has been the standard in prototyping electronics due to its wide variety of functions, ease of programming, and cost. However, much like any other piece of technology it is becoming outdated by faster, smaller and cheaper microprocessors such as the pro-trinket model released in 2014. (A previous article I wrote linked here highlights the key advantages of the Trinket over the Uno.) Fast forward to early 2016 to the release of adafruits most current line of development microcontrollers, called “Feathers” with a host of companion shields referred to as “Featherwings“, where these processor boards now come integrated with certain features such as WiFi connectivity, Bluetooth, LoRa radio and Shields that allow for RTC data logging, Ethernet connectivity and various display interfaces. The Objective of this article is to discuss the two major types of processors used within the Feather line of microprocessors.
Materials and Methods
Featherwing 32u4 v.s Featherwing M0
Which is best? Well, it depends on a host of things. To begin let’s start with a side by side comparison with specs, cost and common troubleshooting problems that I and others in the maker community have encountered. Differences I found especially imporant are denoted with an *.
At the Feather 32u4’s heart is at ATmega32u4 clocked at 8 MHz and at 3.3V logic, a chip setup that has long been tried and tested. This chip has 32K of flash and 2K of RAM and is very comparable of that of a basic Arduino Uno (excluding the use of EEPROM memory).
At the Feather M0’s heart is an ATSAMD21G18 ARM Cortex M0 processor, clocked at 48 MHz and at 3.3V logic, the same one used in the new Arduino Zero. This chip has a whopping 256K of FLASH (8x more than the Atmega328 or 32u4) and 32K of RAM (16x as much)!
Measures 2.0″ x 0.9″ x 0.28″ (51mm x 23mm x 8mm) without headers soldered in
Light as a (large?) feather – 4.8 grams
ATmega32u4 @ 8MHz with 3.3V logic/power*
32K of flash and 2K of RAM*
3.3V regulator with 500mA peak current output
You also get tons of pins – 20 GPIO pins
Hardware Serial, hardware I2C, hardware SPI support
7 x PWM pins
10 x analog inputs (one is used to measure the battery voltage)
Built in 100mA lipoly charger with charging status indicator LED
Pin #13 red LED for general purpose blinking
Power/enable pin
4 mounting holes
Reset button
Long Battery Life*
From a purely technical perspective, it is clear that the ATSAMD21G18 ARM Cortex M0 processor is a superior processor in nearly every regard. However, running such a robust processor comes at the cost of power consumption which is likely an important factor in the design of whatever project you are looking at.
Ease of use and support
While specs are an important aspect to consider when selecting which type of feather you are going to work with another key component is the technical support available for the device. Like any new piece of technology, these adafruit products are not without their pitfalls, manly in the support of preexisting header libraries. I have found in my personal experience that the 32u4 model of the feather which uses an older processor found in many Arduino products has much more firmware support and has the ability to repurpose existing Arduino sketches much easier than with the M0.
The ATSAMD21G18 ARM Cortex M0 processor is much newer and more robust, which comes with the issue of not YET having support for many of adafruit and Arduino libraries. However, if your project doesn’t rely on interfacing with pre-existing technology or have dependencies on the old software you will probably be fine.
Discussion
In short, if you are doing projects that otherwise would have been prototyped on a device like an Arduino that doesn’t require a lot of processing power or large complicated scripts and battery consumption is of importance, the best choice would be the Feather32u4.
For more sophisticated projects that don’t require a lot of external dependencies, and for those with a strong background in computer and microprocessor programming, the feather m0 with ATSAMD21G18 ARM Cortex is a workhorse that allows for some pretty sophisticated programming with an abundance of memory and processor speeds.
Troubleshooting Some Common Problems
I “did something” and now when I plug in the Feather, it doesn’t show up as a device anymore so I cant upload to it or fix it…
No problem! You can ‘repair’ a bad code upload easily. Note that this can happen if you set a watchdog timer or sleep mode that stops USB, or any sketch that ‘crashes’ your Feather
Turn on verbose upload in the Arduino IDE preferences
Plug in feather 32u4/M0, it won’t show up as a COM/serial port that’s ok
Open up the Blink example (Examples->Basics->Blink)
Select the correct board in the Tools menu, e.g. Feather 32u4 or Feather M0 (check your board to make sure you have the right one selected!)
Compile it (make sure that works)
Click Upload to attempt to upload the code
The IDE will print out a bunch of COM Ports as it tries to upload. During this time, double-click the reset button, you’ll see the red pulsing LED that tells you its now in bootloading mode
The Feather will show up as the Bootloader COM/Serial port
The IDE should see the bootloader COM/Serial port and upload properly
The spatial and temporal distributions of CO2 at the earth-atmosphere interface and at the boundary layer above it are of great importance for soil, agricultural and atmospheric changes. In general, the source of elevated CO2 concentrations at this boundary layer depends upon root respiration and soil microbial activity (Blagodatsky and Smith, 2012; Buchner et al., 2008; Nakadai et al., 2002).
Improvement of CO2 gas concentration detection could lead to a better understanding of the agricultural cycle and the transport mechanisms at this critical interface. Moreover, better monitoring methods to detect CO2 concentrations can provide improved modeling and decision-making to enhance agricultural productivity. Until recent years, the techniques for detecting CO2 concentrations included only stationary sensors that give close-range readings or the use of remote sensing platforms, which have limited accessibility due to their high costs and relatively low resolution (Toth and Jóźków, 2016). In addition, the use of remote gas sensing is usually inapplicable inside greenhouses.
The growing use of open open-source hardware (e.g., Arduino) for research purposes creates new opportunities to bridge the gap between CO2 detection at high resolution and affordability. In agriculture, the use of open-source hardware and sensors for CO2 gas sensing is still novel with little supporting research. However, recent research has revealed the possible applications of operating such gas sensing systems for agricultural purposes. For example, the use of an open-source gas sensing device to detect CO2 concentrations in a rectangular greenhouse (106 m × 47 m) was recently proven by Roldán et al. (2015). Using a low-cost electrochemical CO2 sensor, they provided a high-resolution CO2 concentration map inside a greenhouse. Their device also had temperature, relative humidity (RH) and luminosity sensors. All of the data from these sensors can be used for better greenhouse operation decision-making..
The aim of our research is to evaluate CO2 concentrations and dynamics using new CO2 (NDIR technology) and O2 (UV technology) sensors inside a greenhouse. We aim to integrate these gas sensors, together with temperature, RH, and luminosity sensors, on a single logging device to gain high spatial and temporal resolution of the greenhouse environment. In addition, we aim to make the device small and transportable to eventually deploy it on a drone or a rail system to get location-tagged data throughout the greenhouse.
2. Materials and Methods
The sensors used in this project are detailed on Table. 1. All sensors were chosen according the following guidelines:
(1) Sensor accuracy similar to that of a laboratory sensor. For example, the CO2 sensor should have the same accuracy range as other CO2 sensors that were reported in recent published studies.
(2) Only off-the-self, open source sensors.
(3) The sensor should be low-cost compared to other existing sensors.
(4) Only sensors that are modular and light-weight, such that the complete system will be light-weight.
Table 1. Details of the device sensors. The “Name/Model” column includes links for each sensor’s web site. Total cost of sensors (without the optional GPS Logger Shield) was less than 300 $.
[ coming soon ]
3. Results and discussion
System integration included all the sensors detailed in Table. 1. Picture of the device and connection scheme are detailed in Fig. 1-A and B, respectively. Two power connections were tested: a 9V battery and a fix connection to wall power using a 9V adapter. The advantage of using the 9V battery is the mobility of the device; however it was sufficient only for ~5 hr when logging data at 5-min intervals.
Fig. 1. Device description, (A) a photo of the device in the laboratory, (B) connection scheme of the different sensors.
To test the device, data was logged for six days in the laboratory (Fig. 2). Overall, measured parameters were as expected. During daytime the temperature was higher than nighttime with lower RH values. Correlation was observed between the working hours in the laboratory (08:00-17:00) and the peaks in luminosity and CO2 measurements. Once the first laboratory member entered the laboratory and turned on the light, we observed a corresponding increase in the luminosity sensor data. In addition, the CO2 in the laboratory air increased slowly due to laboratory member’s respiration, reaching almost 1,000 ppm. During Saturday and Sunday (8/12/17-8/13/17) the laboratory was closed, therefore luminosity decreased to zero and CO2 equaled atmospheric values (~400 ppm). O2 measurements were considered as problematic because of their high dependency on temperature. O2 concentrations range from 20.5% to 20.8%, however the oscillations were related to the temperature changes and not to changes in absolute values. Each time the laboratory air reached a minimum in temperature, O2 reached a maximum and vice-versa (Fig 3). Different attempts to calibrate the O2 sensor to the temperature were not successful, including using the ideal gas law(e.g., VAISALA, 2012) and talking with the manufacture (“CO2Meter”).
Fig. 2. Time series results from six days in the laboratory.
Fig. 3. Temperature effect on the O2 measurements.
Second deployment of the device was in an agricultural greenhouse located in Oregon State University, Corvallis, Oregon. The device was installed in the end of the greenhouse; ~5 m from the window and ventilation entrance (Fig. 4-B). To achieve reliable measurements, direct sunlight on the sensors was blocked using a cover installed 4 cm above the device. The cover was only above the device, such that air could freely circulate between the device and the surrounding greenhouse air. The only sensor that was exposed to direct sunlight was the luminosity sensor. Taking measurements in the shade is considered as standard procedure (for more details, see the “American geoscience institute” website). Moreover, shade is essential for the CO2 sensor because this sensor uses infra-red technology (radiated heat energy); thus exposing it to direct sunlight will bias the measurements by changing the infra-red energy in the sensor.
Results from the greenhouse measurements are shown in Fig. 4-A. Temperature and RH had the same typical daily cycles as shown in the laboratory data. During daytime, air temperatures were higher with lower RH values compared to nighttime. We note that the Y-axis scales are different between the greenhouse and the laboratory because of higher diurnal cycles in the greenhouse (no AC was used). Luminosity sensor reached full saturation (40,000 lux) during daytime, which indicated direct sunlight inside the greenhouse. Two reasons can explain the “noisy” luminosity readings shown in Fig. 4-A. First, the presence of clouds that temporarily decreased the light entering the greenhouse, and second, the fact that the device was measuring only at a single point below the plant leaves, thus the reading were subject to shade influence according to the angle between the sun and the leaves above the device. CO2 had a small diurnal changes in the scale of a few tens of ppm. These oscillations were within the sensor accuracy limit (± 30 ppm ± 3 % of measured values), and therefore cannot be observed using this sensor model. We note that the experimental greenhouse that was tested here was only with a low density of plants (i.e., number of plants per greenhouse area). In high-dense greenhouses, such as commercial tomato greenhouses, the overall photosynthesis will be much greater, causing CO2 daily oscillations above 100 ppm and thus the CO2 sensor will be more relevant. O2 dependency on temperature was the same as in the case of the laboratory measurements and not because of actual changes of the O2 in the greenhouse air.
Fig. 4. (A) Time series results from five days in the greenhouse, (B) a picture of the greenhouse taken from the entrance door.
We also tested the use of a GPS Logger Shield to get position readings inside the greenhouse. In this case, the shield replaced the real time clock and the microSD board. The accuracy of the GPS position is highly dependent on the greenhouse roof material. In our case, the position readings were not stable with maximum offset of 32 m compared to the true device position inside the greenhouse. Results of the GPS readings are shown in Fig. 5-A were each re point is a single position reading and the green point is the true device position. The number of satellites for each reading is shown in Fig. 5-B.
Fig. 5. GPS experiment, (A) GPS positing readings from one fix position inside the greenhouse (green symbol marks the true position), (B) number of satellites in each reading.
4. Summary
Integration of the different device sensors was successful and two 5-days measurement periods were conducted inside a laboratory and a greenhouse. Main conclusions at this stage are: (1) temperature, RH, and luminosity sensors were reliable and in the desired accuracy range, (2) problematic dependency of O2 sensor with temperature, (3) CO2 accuracy was not sufficient to measure the daily CO2 oscillations inside the experimental greenhouse.
We are now focusing to achieve the following objectives until the end of 2017: (1) improving the code for better power consumption; (2) validation of our CO2 sensor against a standard laboratory CO2 sensor that is frequently used in academic studies – IRGAs GMD-20 manufacture by Vaisala; (3) deploying the system on a drone or on a HyperRail device to obtain spatial greenhouse data and not only fix point measurements. The HyperRail is a low-cost rail system originally designed for moving a hyperspectral camera inside a greenhouse (additional information can be found in this link); (4) finalizing a short paper on the integration and deployment of the system in a greenhouse, which is now in preparation.
5. References
Blagodatsky, S., Smith, P., 2012. Soil physics meets soil biology: Towards better mechanistic prediction of greenhouse gas emissions from soil. Soil Biol. Biochem. 47, 78–92. doi:10.1016/j.soilbio.2011.12.015
Buchner, J.S., Šimůnek, J., Lee, J., Rolston, D.E., Hopmans, J.W., King, A.P., Six, J., 2008. Evaluation of CO2 fluxes from an agricultural field using a process-based numerical model. J. Hydrol. 361, 131–143. doi:10.1016/j.jhydrol.2008.07.035
Nakadai, T., Yokozawa, M., Ikeda, H., Koizumi, H., 2002. Diurnal changes of carbon dioxide flux from bare soil in agricultural field in Japan. Appl. Soil Ecol. 19, 161–171. doi:10.1016/S0929-1393(01)00180-9
Roldán, J., Joossen, G., Sanz, D., del Cerro, J., Barrientos, A., 2015. Mini-UAV Based Sensory System for Measuring Environmental Variables in Greenhouses. Sensors 15, 3334–3350. doi:10.3390/s150203334
Toth, C., Jóźków, G., 2016. Remote sensing platforms and sensors: A survey. ISPRS J. Photogramm. Remote Sens. 115, 22–36. doi:10.1016/j.isprsjprs.2015.10.004
VAISALA, 2012. How to Measure Carbon Dioxide [WWW Document].
An RGB light sensor we evaluated for use in the Evaporometer project. This sensor measures RGB, color temperature, lumens, and visible light spectrum, and has features that currently aren’t useful for our project, but are still worth documenting in this post.
Setting up the sensor:
The TCS34725 was connected to an Adafruit Feather 32u4 using traditional I2C, although the RGB breakout features an additional port for a RGB LED to mimic the RGB values read by the sensor. More on the full functionality can be found on Adafruit’s website here. An SD card breakout was also connected to store the RGB, color temperature, lumens and visible light spectrum values.
A sensor right next to the white LED reads the RGB values of light reflected by different objects.
Testing the sensor:
For this test we were most concerned with measuring RGB values. Color could be read by reading the different colored light that was reflected by an object. A built in white LED is used to present a consistent light source for the sensor to use. This white light must be powered on in order to produce readings and RGB readings can only be made when objects are touching or placed directly in front of the sensor. Our test sketch, which can be used on Adafruit and Arduino dev. boards exists here.
With the LED light powered on, the TCS34725 requires 5 mA of power to operate, making it an unfavorable candidate for long term deployment.
Evaluation of the TCS34725:
After testing Adafruit’s TCS34725 and running a current test, it was clear that the functionality and power needs of this sensor were not what we were looking for on the Evaporometer project.
A significant amount of power would be needed to read the RGB values and such data would be virtually useless in spacious outdoor areas. While lux, color temperature, and the visible spectrum could be used to compare data with our Evaporometer’s TSL2591 sensor, it also makes sense to integrate a second TSL2591 sensor or conduct a side-by-side comparison of the two sensors to see if programming the TCS34725 to only measure lux values requires less power than the TSL2591.
UPDATE: A Closer Look at TCS34725 Functionality
As I mentioned at the end of the previous section, I thought this sensor could be useful to us so long as we disabled the RGB color reading functionality and shut off the white LED to save power. After some digging around in the TCS34725 data sheet and TCS34725 library I learned a lot more about the full functionality of this neat light sensor – like many Adafruit sensors, this RGB sensor is capable of entering a low power mode that drastically reduces current consumption by only reading RGB color info at specified times. Unfortunately, I also found that forgoing measurement of RGB values also makes it impossible to calculate Lux (lumens per square meter). In this version of the TCS34725 library, lux calculations are derived from the RGB values of the light sensor, however this comment in the source code suggests that this may change:
The developers suggest using the clear (C) measurement to find lux – hypothesizing that doing so may improve accuracy. While the clear value is also measured in the same function that measures RGB values, it is possible an updated version of the calculateLux function and a well timed interrupt could also get the job done (using more power than our preferred IR/Full light sensor).
– Marissa Kwon, URSA and NSF Grant Summer Student Researcher
So far we have used this series of blogposts to discuss a lot of the technical details about how certain processes are happening within the Evaporometer Transmitter and Receiver. A series of diagrams has been made to help visualize how everything comes together. First will be a diagram showing how all the sensors are connected to the Feather 32u4 and battery, followed by two flowcharts illustrating how abstract environmental conditions are transformed into the data logged in our spreadsheet.
Evaporometer Transmitter Connections:
Pictured above is a very simple assembly diagram for the Evaporometer Transmitter. Notice that many of the devices have the same yellow and blue colored lines – these are the serial clock and data lines respectively and they are all hooked up to the same pins on the Feather. The red and black lines represent power and ground, and all devices must be connected to these nodes in order to receive power. In real life, the power and ground rails exist along the far edges of a protoboard or breadboard so that all the device wires do not need to be crammed into one small area.
Flowcharts Following the Progression of Evaporometer Data:
Sensors turn information collected from the environment into quantifiable values such as integer and float values that eventually get passed on to the receiver and logged onto a spreadsheet where we can see them online. The diagram below shows how data moves from one component to the next through the Evaporometer Transmitter. Each labeled point on the flowchart indicates where the data must undergo at least one change before being able to proceed to the next.
Although it may seem like a fairly simple process, the sensor data must undergo several transformations before it can reach its destination on the spreadsheet. The integer and float values collected by the sensors must be turned into strings so they can be concatenated into one long packet of data. Now as a single piece of information, the string is cast into an array of 8-bit characters so that it can be transmitted and received using the LoRa components built into our micro-controllers. When the message is received it must also be broken back into the same pieces before it was concatenated and able to be cast back into the float or integer values. To do this we add a comma as a delimiter placed in between data points during concatenation, and once it is received we use use strtok() to divide up our character array at the location of the commas to separate the packet back into different pieces of data.
The diagram below illustrates each process that takes place in the transmitter, starting with data collection right after the RTC alarm wakes it up.
– Marissa Kwon, URSA and NSF Grant Summer Student Researcher
The description of process of using PushingBox and the Google scripts was updated 10/3/18 (primarily with regards to the usage and interfacing with the Google Script). The most up to date and comprehensive tutorial is now maintained here.
Abstract
Connecting… Connecting… Transmission Received!
After months of prototyping, experimenting and redesigning we have successfully deployed our first, “fully open source” transmitter and receiver with near real time updates to a google spreadsheet. This post will give a step by step procedure on how to set up all the software and web interfaces for this project.
Step One – Creating and Deploying your Google Sheet
A) Log in to your free Google account and create a new “Google Sheet,” this is the spreadsheet which will be populated by the sensor data received by the Ethernet Featherwing data logger. At this point, you will need to copy and save your spreadsheets URL key for use in your Arduino code. This key is listed in the URL between the “/d/ ” and the “/edit ” of your new spreadsheet (see red box).
B) Name your Spreadsheet something original like “Environmental Data,” now we must create our Google App Script (similar to JavaScript) which will process our data and populate our spreadsheet correctly. To insert our Google App Script we first navigate from our spreadsheet to the Script Editor, found under the Tools tab:
Tools →Script Editor
We will now be on the Google Script Editor page. Now we can name our Gscript something extra-creative like “My Environmental GScript”, but do not have to. Fortunately, we do not have much that we have to do on the Script editor page.
1) Copy and Paste the Google Script code into the Editor. (link to GitHub with all needed codes here)
2) Save the script: File→Save All
3) Deploy as a web App: Publish-→ Deploy as web App…
C) Our final Step for setting up our Google sheet will involve configuring the Web deployment parameters. It is important you set all these fields correctly. If you don’t set all four fields exactly your spreadsheet won’t work properly (trust me…). This step takes place immedietly after clicking “Deploy as Web App”.
Field 1) Copy and Save the “Current web app URL:“ which has just been generated, we will need this later when we configure our API in PushingBox.
Field 2) Save a project version as “new” on each iteration, it’s important to note that if you don’t make a new project version for each script revision you make (if you decide to make any revisions), your script revisions won’t be updated on the web. This is counter-intuitive, and easy to neglect, but this is currently how Google has this system configured.
Field 3) “Execute the app as: “ set this to “me(your GMail here) ”.
Field 4) “Who has access to the app: “ set this to “anyone, even anonymous.” *This is important, and allows Pushingbox access the spreadsheet.*
Step Two – PushingBox API
Pushingbox.com serves as simple, free, and easy Application Programming Interface (API) middleman in allowing our Evaporometer data to be palatable to Google Sheets. The need for using the PushingBox API intermediary is to turn our HTTP Ethernet transmitted data into Google compliant HTTPS encrypted data. Although other programmers have found workarounds to this system (found here) using PushingBox is easier, and we are given up to 1,000 requests (Data Transmissions) a day.
*1,000 push requests is more than enough for updates every 2 minutes in a 24-hour cycle but not enough for minute by minute updates (thus the “” around real time) but this service is completely free.*
Below are the steps for configuring Pushingbox to work with our data:
1. Create a PushingBox account using your Gmail.
2. At the top click on “My Services”
3. While now in “My Services” go to the “Add a service” box and click on “Add a service”.
4. The service you will want to create is the last item on the list of services and is called: ”CustomURL, Set your own service !”. Now select this CustomUrl service.
5. A box will open up and ask for three items. Fill them out as below, then hit submit
Name of your CustomURL configuration:
Root URL:This url will start with https://script.google.com… as this is your Google Script address saved from Part 1
6. After submitting your service you must now create a scenario for the service. To do this select “My Scenarios” at the top of the page.
7. Enter an appropriate name for your scenario in the “Create a scenario or add a device” box. After you name your service click the “add” button to the right.
8. Now it will ask to “Add an action for your scenario” You should now choose the “Add an action to this service” button listed next to the service name you created in the previous step. This assigns your new scenario to your new service.
9. A data box will open asking you for your “Get” or “Post” method (we use the Get method although this seems counter-intuitive). For recording all our Evaporometer data into your Google sheet you need to link your variable names listed in your Google App Script to the names listed in our Arduino sketch. Formatting the names correctly in the Pushingbox API will accomplish this linking task.
Copy and paste the following string into your box.
?key0=$key0
amp;val0=$val0
amp;key1=$key1
amp;val1=$val1
amp;key2=$key2
amp;val2=$val2
amp;key3=$key3
amp;val3=$val3
amp;key4=$key4
amp;val4=$val4
amp;key5=$key5
amp;val5=$val5
amp;key6=$key6
amp;val6=$val6
amp;key7=$key7
amp;val7=$val7
amp;key8=$key8
amp;val8=$val8
amp;key9=$key9
amp;val9=$val9
amp;key10=$key10
amp;val10=$val10
amp;key11=$key11
amp;val11=$val11
amp;key12=$key12
amp;val12=$val12
amp;key13=$key13
amp;val13=$val13
amp;key14=$key14
amp;val14=$val14
amp;key15=$key15
amp;val15=$val15$
Note: STATEMENT BEGINS WITH ” ? ” TO INDICATE “GET”
The result should look like as follows (roughly, as the data string shown in the image is slightly out of date, but the rest is correct), but with your own scenario name and Device ID number:
*Make sure to copy your “DeviceID” string that will be generated after setting up the service, you will need it for both preliminary testing in the next step, and later for the Arduino Sketch in Part 4.*
Step Three- Testing API and Google Spreadsheet
Before we move on to the last step, in which we program the Adafruit Feather 32u4 with LoRa Radio Module and complimentary Ethernet Feather Wing which will log data through the web, it would be helpful to test that everything we have done thus far is correct. If we wait to complete the hardware portion then the cause of any errors may be more difficult to track down. Fortunately, we have a simple method of testing our code so far. We can just directly enter some hard-coded pseudo-data into our web browsers address bar and check that our Google sheet is being updated correctly. Here is an example of what you might copy and paste into your browsers address bar.
If you like you can even re-enter new fake data with different values for subsequent rows, however, remember you only have a 1,000 requests per day from Pushingbox.com so don’t go crazy! If this procedure did not update your spreadsheet then go back and review Parts 1 -3 of this instructional for errors before attempting Part 4. Congratulations if you have made it this far, the worst is over! (it took me about a dozen times to get all the previous steps correct) Update – the modification of this process and the Google Script since the initial writing of this blog post were primarily to simplify these steps
Step Four – Hardware Set-up and Arduino Sketches
After confirming you were able to push some hard-coded pseudo-data directly from your browser onto your Google Sheet you are now ready for the next step which involves sending data directly from the Feather 32u4 and complimentary Ethernet Feather Wing to Google via Pushingbox over an Ethernet connection. Before you can upload and run the included Arduino sketch to the Feather Board you must complete the following three steps.
Hardware
1. Wiring the Feather 32u4 board to the Ethernet FeatherWing: a guide on how to do this can be found on the adafruit website here, as well as one of my previous posts, found here.
Software(1):
2. There are a few steps to set up your Arduino IDE to work with your Feather 32u4 board.. These steps include downloading the necessary libraries and configuring the Arduino IDE (1.6.4 or later) with the correct Feather Board package. Many of you will have already installed all the software you need, for those that have not, below are links to retrieve the needed libraries and the Feather 32u4 board support package.
The two required libraries for our particular sketch:
Two great tutorials to walk you through getting the correct Feather 32u4 board and Ethernet FeatherWing package installed into the Arduino IDE if you haven’t done this already:
3. You are now ready for the final step of the project, you need to copy and paste the included Arduino sketch, customize it for your device, and then upload the sketch onto your Feather 32u4 board and Ethernet FeatherWing. Update – that sketch is now out of date, refer to the Loom Library instead for the corresponding updated code and process.
A) Your MAC Address (Found in Ethernet packaging)
B) Your Static IP address (May need to be given by IT admins)
C) Your Pushing box Device ID (devid)
That’s it, you’re done! If everything was completed correctly your output will look something like the picture below:
Populated Google Spreadsheet (left) and Ardiuno IDE with example code.
Final Receiver and Ethernet data logger set-up.
– Tom DeBell, Beginning Researcher Support Program Researcher
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
