💾 Archived View for benjaminja.info › log › 2024 › 02 › 07-station_api captured on 2024-03-21 at 15:01:18. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
2024-02-07 | @ttocsneb
For my final semester of college, I took a special projects course which is essentially a research class, but instead of the professor telling me what to do, I go to the professor with a project that I would like to do. If they agree, they act as my mentor to help me plan out and follow through with the project.
I decided that I would like to finish my weather station once and for all. At this point, I have already finished the hardware portion of the project (though there are some issues I have yet to work out). My proposal was to build a web server that would link between the weather station hardware and the end user.
I took this idea to my professor and he liked the idea. Most of it was already thought out, but not really planned out. He was able to help guide me with different ideas on how I could implement what I want to do.
This document is the weather station API for my project. Keep in mind that it was written during the planning phases of the project and so some of the plans have changed since writing.
Originally written on September 26, 2023
The weather station will be split up into several parts. This document will provide how each module works and how they're structured.
There are a total of 4 modules: Station, MQTT Broker, Weather Server, and UI Server.
The station will collect live weather data and pass it to the MQTT broker. The broker will route station packets to any listening weather servers. The weather servers will store a history of weather data as well as handle requests from UI servers. UI servers display information directly to the client.
Stations can be in locations where it is not possible to make requests from outside the local network. MQTT is a protocol that is designed for IoT systems. MQTT clients may publish/subscribe to endpoints. This allows for stations to send weather updates to all listening weather servers. Weather stations would also be able to make requests to specific stations if needed.
There will be three primary endpoints for stations.
Definitions:
QoS (Quality of Service) determines how the message is sent to subscribers. There are 3 levels of quality: 0 (at most once), 1 (at least once), 2 (exactly once). With QoS 1 and 2, the message will be temporarilly stored on the broker and makes sure that subscribers get the message.
Stations publish their weather updates to this endpoint. Weather servers may either subscribe to individual station's weather endpoints or they could subscribe to `/station/weather/+` to get every station's weather updates.
Stations publish to this endpoint automatically. The period should be once a minute.
This should have a Quality of Service 0 (at most 1 received message)
The content will be structured as follows:
type: object
properties:
time:
description: >
Time of the sample.
Should be formatted with ISO-8601 (2023‐09‐14T14:01:26-07:00)
type: string
id:
description: >
id of the station
type: string
additionalProperties:
description: >
There are a limited number of sensor ids. Multiple sensors for each type are
supported.
* winddir
* windspd
* windgustspd
* windgustdir
* windspd-avg2m
* winddir-avg2m
* windgustspd-10m
* windgustdir-10m
* humidity
* dewpoint
* temp
* rain
* dailyrain
* barom
* soiltemp
* soilmoist
* leafwetness
* solarradiation
* uv
* visibility
* indoortemp
* indoorhumidity
type: array
items:
type: object
properties:
unit:
type: string
enum:
- mph
- mps
- c
- f
- in
- mi
- nm
- mi
- km
- inhg
- torr
- hpa
- '%'
- deg
- rad
value:
type: number
required:
- time
- id
Stations would publish information about themselves to this endpoint. Info updates would only occur if inquired through the `/station/request/{id}` endpoint.
This should have a Quality of Service 1 (at least 1 received message)
The contents will be structured as follows:
type: object
properties:
make:
description: Name of the manufacturer
type: string
model:
description: Name of the hardware model
type: string
software:
description: Name of the software
type: string
version:
description: Software version
type: string
latitude:
description: Latitude of the station
type: number
longitude:
description: Longitude of the station
type: number
elevation:
description: Elevation above sea level of the station
type: number
district:
description: Neighborhood/District the station resides in
type: string
city:
description: City the station resides in
type: string
region:
description: State/Province/Region the station resides in
type: string
country:
description: Country the station resides in
type: string
rapid-weather:
description: Whether the station supports rapid-weather updates
type: boolean
Weather servers may make requests to stations by using this endpoint. Multiple types of requests can be made here.
This should have a Quality of Service 1 (at least 1 received message)
type: object
properties:
action:
description: The action that is being requested
enum:
- info
- rapid-weather
required:
- action
Stations may update the weather more often than once a minute. This would be primarilly for clients actively watching a specific station. Because of this, it is likely that there will be no servers listening to this endpoint at any given time. To limit unecessary network traffic, a request needs to be made to enable the rapid-weather endpoint. The rapid-weather endpoint should be active for 1 minute after the latest request.
This should have a Quality of Service 0 (at most 1 received message)
This uses the same payload schema as `/station/weather/{id}` .
There are several MQTT brokers available, most support TLS, http-ws, https-ws. For this implementation, Mosquitto will be used as the broker over plain text. This is because everything will be self contained in one computer.
For public Station MQTT Brokers, TLS should be used to allow for authentication and to prevent clients from writing to endpoints they do not have rights to.
The weather server will store a history of weather data and distribute it to clients. Clients can request for weather at a specific location, or from a specific station.
The weather server can be configured by admins to listen to multiple station MQTT brokers. The weather server will listen to all publishing stations on each registered broker.
Since stations may be added to the MQTT server at will, the weather server will find itself getting messages from stations it does not recognize. The server will create a stub station for this new data and make a request to get info on the station. This info will be populated into the database.
Stations that are removed from MQTT servers cannot be discerned from stations that have gone offline. So station data will not be removed unless by an admin.
Stations that have not made updates after a certain amount of time will be considered offline. These stations will not be discoverable and data will not be used. Clients may still get history on these stations if they know the station id.
A detailed history will be kept for the previous 24 hours. This will be one sample per minute per station. After this, a less detailed history will be stored for the previous 7 days. This will be one sample per hour. Samples will be averaged together. After this one sample per day per station will be saved. Data will be averaged together as well as include mins/maxes.
There will need to be an account system so that administrators can manage the server.
The weather server would have access to many stations. A client really only needs to know the weather of a few stations nearby their location. The client can get the weather of the nearest stations in an area with a weighted average.
/location/conditions/
🖼 Map of stations to be averaged and their distance weights
A client could also get the closest station to their location. From here they would be able to view the station's weather.
/location/nearest/
Sometimes you want to know the weather for a location where you are not. Getting weather for a city name can be useful. The weather would be averaged among all stations in the region.
/region/conditions/
If you own a station, you will generally want to be able to view the conditions of that station. You can follow your station and have a custom view of the station at all times.
/user/{id}/settings/
/station/{id}/conditions/
/station/{id}/conditions/history/
/station/{id}/conditions/updates/
/station/{id}/conditions/rapid/
The UI server will be written in Go using htmx. It will connect browsers to the weather server api.
The website will be split up into a three sections:
The client will be able to switch between these pages in a few ways. The location view will be the default landing page. Clients may search for a city, or district to get the region page. From the location page, the nearest station can be opened. If a user is logged in, they will also be able to open followed stations from the menu.
There are a few ideas that I like, but feel that now is not the right time to flesh them out.
Having this system be federated in a way that there could be many weather servers dedicated to specific regions would be interesting. This could be done by having a list of publicly available servers and the regions they cover.
It will also be important to have weather prediction capabilities. An api should be designed so that prediction services could collect historical data and run some model to give estimates for either specific stations or general regions.
There are now 11 weeks left until the final day of class. The plan for this project will need to be set aside carefully. I have provided some rough estimates, but expect that they may be off by a bit.
The first step will be to build the station server and connect it to an MQTT server. A simple server will be made that will monitor the MQTT endpoints and be able to send messages back to the station server. This server will be the start of the weather server.
I would anticipate this to take 2 weeks to complete.
The next step will be building out the weather station. This will be done by setting up the database and handling any internal logic. This would include discovering stations, logging weather samples, consolidating samples, and getting rapid updates.
This should take about 2 weeks to do.
Basic endpoints will be created on the weather server, and the ui server will be started. This will be the MVP of the weather station. Being able to view live weather data, charts of the past week's weather, and get real time updates.
This would take 4 weeks to do. This milestone could be split further into station, location, region, and history.
The final 3 weeks will be spent getting an account system setup. This will include all of the non-essential parts of the system, but very important to become production ready. Administration, account creation, email, and account management.
Since it is possible that a milestone may become delayed, and features will need to be dropped, these final components will naturally be the first to go.