Opened 11 years ago
Closed 10 years ago
#529 closed task (fixed)
Retrieve and store temperature and humidity measurements
Reported by: | Nicklas Nordborg | Owned by: | olle |
---|---|---|---|
Priority: | major | Milestone: | LabEnv v1.0 |
Component: | net.sf.basedb.labenv | Keywords: | |
Cc: |
Description
New 'climate' sensors have been installed in the lab and they are connected to the network! It would be nice if Reggie could query the sensors at regular intervals and store temperature and humidity on the server.
Change History (43)
comment:1 by , 11 years ago
Summary: | Retreive and store temperatore and humidity measurements → Retreive and store temperature and humidity measurements |
---|
comment:2 by , 11 years ago
Owner: | changed from | to
---|
comment:3 by , 11 years ago
Background information:
- A number of climate sensors have been purchased and installed at the department of Oncology at Medicon Village/Lund University. Four sensors have been purchased, but only no. 1,2, and 4 were working at the end of February, 2014. On 2014-03-31 finally sensor no. 3 was on-line, but sensor no. 1 had then been off-line for unknown reasons for some days.
- The sensors are of type Thermo Recoder TR-702W from ThermoWorks. For product information, check http://www.thermoworks.com/products/logger/t-and-d-tr700series.html. They can be configured to be accessed over the network from a web browser, with a page title indicating the lab/room a sensor is placed in.
- The web page shown, when a sensor is accessed over the network with the standard URL, displays current as well as older values of temperature and humidity (in units of °C and %, respectively). Current values are displayed as numbers, while a graph shows older values (the numerical values for a specific time can be found by placing the mouse pointer over the graph at the point of interest). The web page is customizable to a large degree by a number of buttons, and the graph can be zoomed by dragging the mouse over a time region. One button displays current settings.
- With all flexibility of the current sensor web page, there are still some limitations. One is that the graph cannot display more than about 12 hours of measurements, making it hard to check changes in the lab environment over longer time periods.
- A specification
man_format-xml-122-all-eng.pdf
of the data format used by the sensors for sending data can be downloaded from http://www.tandd.com/support/download/manual/xml_format.html (no direct link available).
Data for communicating with the sensors are as follows (2014-04-02):
URL | Title | Serial Number | IP-number | Comment |
http://givare1.onk.lu.se | G1 Maskinrummet (B32C2) | 48740059 | 130.235.5.197 | On-line |
http://givare2.onk.lu.se | G2 RNAlab (B33A2) | 4874007D | 130.235.5.198 | On-line |
http://givare3.onk.lu.se | G3 Apparatrummet (C21B3) | 48740075 | 130.235.5.199 | On-line |
http://givare4.onk.lu.se | G4 Lilla apparatrummet (C22A3) | 4874007B | 130.235.5.200 | On-line |
comment:4 by , 11 years ago
Retrieval of sensor data:
Inspection of the sensor web page code and the data format specification in document man_format-xml-122-all-eng.pdf
mentioned above, revealed the following (Nicklas Nordborg gave invaluable help here):
- Adding "
/current.inc
", "/params.inc
", or "/datas.inc
" to a sensor URL resulted in the return of different data in a JSON-style format. - Adding "
/current.inc
" to a sensor URL returned JSON-style data, that after Base64-decoding and conversion among other things contained serial number of the sensor, current local time, temperature in °C, and humidity in %. The latter two values were given with an accuracy of one decimal. - Adding "
/params.inc
" to a sensor URL returned JSON-style data, that after Base64-decoding and conversion among other things contained the title of the sensor, indicating the lab/room it was installed in.
To test that the principles stated above were fully understood, a "dummy" application was implemented consisting of a JSP script and a Java servlet. The HTML page with the JSP script contained a pop-up menu to select a sensor plus a button to fetch its data by sending an Ajax request to the servlet. Tests showed that it was possible to retrieve sensor title, serial number, current local time, temperature, and humidity values that were consistent with what was returned from the original sensor web page. All data retrieval and conversion in the dummy test application were performed by the Java servlet, without use of any JavaScript in the HTML page, unlike the original sensor web page.
comment:5 by , 11 years ago
Detailed examples for retrieving sensor data from climate sensors:
Even though all steps are simple, the full procedure to retrieve a sensor value from a climate sensor can be hard to find from scratch, so some examples in Java are therefore described here.
General:
- The raw data content for a url is obtained by a HTTP "get" request, e.g. performed using the Apache Commons HttpClient package. In the example below, the raw data content is contained in String variable "
content
" (output of error messages has been simplified):import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.HttpStatus; import org.apache.commons.httpclient.methods.GetMethod; private String fetchDataFromUrl(String url) throws ServletException, IOException { String content = null; GetMethod getMethod = null; try { getMethod = new GetMethod(url); HttpClient httpClient = new HttpClient(); int statusCode = httpClient.executeMethod(getMethod); // Check status code if (statusCode != HttpStatus.SC_OK) { System.out.println("Error when trying to execute GetMethod: " + getMethod.getStatusLine()); } content = getMethod.getResponseBodyAsString(); } catch (IOException e) { System.out.println("IOException when trying to execute HttpClient method : " + e); } finally { // Release current connection to the connection pool once you are done if (getMethod != null) { getMethod.releaseConnection(); } } return content; }
- The raw data content is processed into a standard JSON-style string by removing odd characters before first brace, and replacing single quotes with double:
String jsonStr = "" + content; // Remove odd character(s) before first left brace "{" int leftBraceIndex = jsonStr.indexOf("{"); if (leftBraceIndex >= 0) { jsonStr = jsonStr.substring(leftBraceIndex); } jsonStr = jsonStr.replaceAll("\\'", "\\\"");
- Retrieval of sensor title, exemplified by sensor URL
http://givare2.onk.lu.se/
:
a.String content = fetchDataFromUrl("http://givare2.onk.lu.se/params.inc");
b. A standard JSON-style stringjsonStr
is obtained by processingcontent
according to step 2 above.
c. The sensor titlelabSensorName
is obtained using the open-source classBase64Coder
(see http://www.source-code.biz for details):// Initialize lab sensor name String labSensorName = ""; try { JSONParser parser = new JSONParser(); JSONObject jsonOutput = (JSONObject) parser.parse(jsonStr); String d30 = (String) jsonOutput.get("D30"); // Decode Base64-encoded data byte[] d30ByteArray = Base64Coder.decode(d30.toCharArray()); // Get sensor name from beginning of Base64-decoded byte array int arraySize = d30ByteArray.length; for (int i = 0; i < arraySize; i++) { char curChar = (char) d30ByteArray[i]; if (curChar != (char) 0x00) { labSensorName += curChar; } else { // Stop after first null byte break; } } } catch (ParseException e) { System.out.println("JSON ParseException e = " + e); }
- Retrieval of sensor serial number, local measurement time, temperature, and humidity values, exemplified by sensor URL
http://givare2.onk.lu.se/
:
a.String content = fetchDataFromUrl("http://givare2.onk.lu.se/current.inc");
b. A standard JSON-style stringjsonStr
is obtained by processingcontent
according to step 2 above.
c. The sensor serial number, local measurement time, temperature, and humidity values are obtained using the open-source classBase64Coder
(see http://www.source-code.biz for details):// Initialize lab temperature and humidity data String serialNumber = ""; String dateTimeStr = null; String temperatureStr = null; String humidityStr = null; try { JSONParser parser = new JSONParser(); JSONObject jsonOutput = (JSONObject) parser.parse(jsonStr); String d20 = (String) jsonOutput.get("D20"); String d21 = (String) jsonOutput.get("D21"); // Decode Base64-encoded data byte[] d20ByteArray = Base64Coder.decode(d20.toCharArray()); byte[] d21ByteArray = Base64Coder.decode(d21.toCharArray()); // Get sensor serial number from Base64-decoded byte array int arraySize = d21ByteArray.length; for (int i = 0; i < 8 && (10 + i) < arraySize ; i++) { char curChar = (char) d21ByteArray[10 + i]; if (curChar != (char) 0x00) { serialNumber += curChar; } } // Get raw variable values from Base64-decoded byte array int ch1 = (d20ByteArray[26] & 0xFF) | ((d20ByteArray[27] & 0xFF) << 8); int ch2 = (d20ByteArray[42] & 0xFF) | ((d20ByteArray[43] & 0xFF) << 8); // Variable nowTime is the number of seconds since 1070-01-01 00:00:00 long nowTime = (d20ByteArray[6] & 0xFF) | ((d20ByteArray[7] & 0xFF) << 8) | ((d20ByteArray[8] & 0xFF) << 16) | ((d20ByteArray[9] & 0xFF) << 24); // Get variable values from raw variable values double temperature = (ch1 - 1000)/10.0; double humidity = (ch2 - 1000)/10.0; // Convert nowTime to number of ms since 1070-01-01 00:00:00 before getting dateTime Date dateTime = new Date(nowTime*1000); // Get variable value strings from variable values DateFormatter dateFormat = new DateFormatter("yyyy-MM-dd HH:mm:ss"); temperatureStr = "" + temperature; humidityStr = "" + humidity; dateTimeStr = dateFormat.format(dateTime); } catch (ParseException e) { System.out.println("JSON ParseException e = " + e); }
comment:6 by , 11 years ago
Design notes:
- A server should be set up to regularly collect temperature and humidity values from the different lab sensors, and store the values in a database. As there is no strong dependence between general steps in the lab work and the lab environment data, the latter should be stored in a different database, possibly using a different (simpler) database management system than is used for BASE/Reggie. However, an API should be created to make it easy to retrieve data from the lab environment database, in order to investigate dependencies between individual lab steps in the lab work and the laboratory environment.
- The lab environment database should store the following data for a lab sensor measurement:
a. Lab sensor number (uniquely coupled to a sensor URL)
b. Measurement date/time
c. Temperature in °C
d. Humidity in %.
- The tests revealed that the response from a sensor came in a small fraction of a second (of the order of 0.2-0.4 seconds), but the time stamps obtained from the responses of the different sensors could differ more than a minute from each other and the system time of the server, where the test program was executed. In order to simplify investigation of environment measurements from different labs at specific time points, it is recommended that the server system time is used, when a time is stored for a measurement.
- Tests also revealed that subsequent sensor temperature measurements a minute apart may fluctuate in a range of 0.5 °C, and the humidity measurements in a range of 0.2-0.5 % units, making detailed comparisons of values within this range be of very limited use. When storing lab environment values, it might therefore be considered to collect measurements during a small time interval (some minutes), and then store the arithmetic mean values. An alternative is to leave it up to the routines displaying or otherwise investigating the values, to make any averaging of the raw data.
- The API for the lab environment database should allow retrieval of data for a chosen sensor number between specified start and end times:
a. If no start time is given, all data for the chosen sensor before or equal to the end time is retrieved.
b. If no end time is given, all data for the chosen sensor after or equal to the start time is retrieved.
comment:7 by , 11 years ago
Summary: | Retreive and store temperature and humidity measurements → Retrieve and store temperature and humidity measurements |
---|
Typo fixed in ticket summary.
comment:8 by , 11 years ago
Design description (initial):
Specific design choices:
- Database management system: The open-source SQLite database management system will be used for storing the lab environment data (see SQLite home page https://sqlite.org for more info).
- In order to reduce instrument noise in stored data, and reduce the number of entries in the database, measurements should be collected during a small time interval (some minutes), after which the arithmetic mean values are stored. The time stored together with a stored data value is set to the mean of the times for the measurements used to calculate the average stored data value.
- Packaging: It was decided to create the new service as a BASE extension, allowing for it to be simply installed from a single JAR file plus configuration file using the BASE extension installation mechanism, available through BASE menu "Extensions -> Install plug-ins/extensions...". After installation, the "Extensions" menu will have a new menu item "Lab Environment", with a number of sub-menus. Creating the service as a BASE extension also has the benefit of keeping the code encapsulated, and allowing it to be installed only at BASE servers, that can take advantage of it (i.e. is connected to ThermoWorks Thermo Recoder TR-702W sensors).
The code has been split up in a number of classes/files, where each one has a more or less well-defined responsibility. The code root directory is named "labenv
" (short for "lab environment").
Files to be installed in specific directories:
File | Directory | Description |
Database file for lab environment values | BASE user files directory defined in configuration file base.config . | File used by the database manager to store the lab environment values. It will be automatically created by the service, if it doesn't exist. Its name is taken from configuration file labenv-config.xml . The BASE user files directory was chosen, as this normally has automatic backup routines coupled to it.
|
labenv-config.xml | Directory where BASE configuration files like base.config are installed. | This directory was chosen as it is nice to have configuration files stored in a single place. |
BASE Service:
File | Description |
labenv/src/net/sf/basedb/clients/web/extensions/service/LabEnvService.java | A service based on the BASE DummyAction service, that repeats an action after a pre-defined time interval. This class combines the functionality of ServiceControllerFactory , ServiceController , and Service , that are implemented as separate classes in more complex services, like the BASE FTP Service. It implements the ServiceControllerAction interface, having public methods boolean isRunning() , void start() , and void stop() . Public method void start() creates an instance of singleton LabEnvironment and stores it as an instance variable. LabEnvironment in turn creates an instance of class LabEnvironmentConfiguration , that reads configuration values from file labenv-config.xml . A java.util.TimerTask instance is created, where the time interval is taken from configuration values, and the public void run method calls public method void measureAndStoreLabEnvironmentData() in the LabEnvironment instance.
|
Lab environment measurements and storage:
File | Description |
labenv/src/net/sf/basedb/labenv/LabEnvironment.java | Singleton class that is the main entry point to the lab environment functionality. It creates an instance of class LabEnvironmentConfiguration , that reads configuration values from file labenv-config.xml . Public method void measureAndStoreLabEnvironmentData() uses utility class LabSensorUtil to measure lab environment values for configured lab sensors and store the values using utility class LabEnvironmentStorageUtil .
|
labenv/src/net/sf/basedb/labenv/LabEnvironmentConfiguration.java | Class that reads configuration values from file labenv-config.xml and stores them in instance variables. Configuration values for individual lab sensors are stored in a list of LabSensorConfig instances. Also has public methods LabSensorConfig findByNumber(Integer number) and LabSensorConfig findByUrl(String url) to obtain the configuration for a specific lab sensor.
|
Lab environment data access objects:
File | Description |
labenv/src/net/sf/basedb/labenv/dao/LabSensorConfig.java | Class for storing lab sensor configuration values in instance variables with corresponding public accessor methods. The values stored are: int number , String url , String name , String startDate , and List<LabSensorAlarmConfig> alarmList .
|
labenv/src/net/sf/basedb/labenv/dao/LabSensorAlarmConfig.java | Class for storing configuration values for a lab sensor alarm in instance variables with corresponding public accessor methods. For future use. |
labenv/src/net/sf/basedb/labenv/dao/LabEnvironmentData.java | Class for storing lab sensor data values in instance variables with corresponding public accessor methods. The values stored are: String sensorUrl , String sensorName , String serialNumber , Date dateTime , long unixTime (number of seconds since 1070-01-01 00:00:00, obtained from dateTime ), double temperature (in °C), and double humidity (in %).
|
Lab environment utility classes:
File | Description |
labenv/src/net/sf/basedb/labenv/util/LabSensorUtil.java | Class with public methods for obtaining values from a lab sensor: String fetchLabSensorName(String sensorUrl) , LabEnvironmentData createLabEnvironmentData(String sensorUrl) , and LabEnvironmentData createLabEnvironmentData(String sensorUrl, String sensorName) . Private method String fetchDataFromUrl(String url) is used to obtain the raw sensor output from a sensor at a specified URL.
|
labenv/src/net/sf/basedb/labenv/util/LabEnvironmentStorageUtil.java | Class for storing and retrieving data in/from the lab environment database. The lab environment database table is created if not existing. The table contains columns for: id INTEGER PRIMARY KEY , sensorNumber numeric , unixTime numeric , temperature numeric , and humidity numeric . Note that sensor measurements are stored by sensor number, not sensor URL or name. The class has public methods ResultSet executeQuery(String instruction) , void executeUpdate(String instruction) , void updateLabEnvDb(int sensorNumber, LabEnvironmentData envData) , void updateLabEnvDb(int sensorNumber, long unixTime, LabEnvironmentData envData) , void updateLabEnvDb(int sensorNumber, long unixTime, double temperature, double humidity) , and List<LabEnvironmentData> queryLabEnvDb(Integer sensorNumber, Date startTime, Date endTime) .
|
labenv/src/net/sf/basedb/labenv/util/ReportTableUtil.java | Class with utility methods for creating statistics for different time intervals. For future use. |
Lab environment servlet and JSP files:
File | Description |
labenv/src/net/sf/basedb/labenv/servlet/LabEnvironmentServlet.java | Java servlet class for obtaining information from lab sensors and the lab environment database. Initially three Ajax commands are implemented: GetLabSensorConfigList , LabEnvironmentSensorQuery , and LabEnvironmentDatabaseQuery .
|
labenv/src/net/sf/basedb/labenv/servlet/LabEnvironmentStatisticsServlet.java | Java servlet class for creating statistics for the lab environment database. Initially two Ajax commands are implemented: labenvironmentdailydistribution and labenvironmentweeklydistribution .
|
labenv/resources/reports/labsensorinfo.jsp | JSP file for obtaining data from selected lab sensor(s). |
labenv/resources/reports/labenvironmentdatabaseexport.jsp | JSP file for obtaining data from the lab environment database, for selected sensor(s) and time period. The obtained data is made available in table format with tab-separated columns, for preview or export to an external file. |
labenv/resources/reports/labenvironmentdatabasestatistics.jsp | JSP file for displaying statistics for the lab environment database, for selected sensor(s) and time period. Initially statistics for daily and weekly distribution of temperature and humidity will be available. |
BASE extension entry points configuration:
File | Description |
labenv/META-INF/extensions.xml | Configuration file for the lab environment BASE extension entry point. Definition of path for extension service class net.sf.basedb.clients.web.extensions.service.LabEnvService , and name of menu item in BASE "Extensions" menu (here a sub-menu with items for individual JSP pages).
|
Miscellaneous files:
- In addition to the files above, a number of other files are needed to make a working extension. A BASE extension may access core BASE classes, e.g. for obtaining directory paths for configuration and user files, but not utilities in other BASE extensions, such as Reggie. Therefore some practical conversion and formatting utilities have been copied from the latter to the lab environment extension.
Conversion files:
File | Description |
labenv/src/net/sf/basedb/labenv/converter/Base64Coder.java | An open-source utility class for coding and decoding data to/from Base-64 format. This class has been used in BASE sister project Proteios SE (http://www.proteios.org/) since 2006. |
labenv/src/net/sf/basedb/labenv/converter/ DateToStringConverter.java StringToDateConverter.java ValueConverter.java | Conversion routines copied from the Reggie extension. |
Files for formatting, layout, etc:
File | Description |
labenv/resources/css/reggie.css | Copy of Reggie CSS file, ensuring a consistent layout. |
labenv/images/ | Copy of Reggie "images" directory, ensuring a consistent layout (only a small sub-set of the image files are used). |
labenv/resources/reggie.js | Copy of Reggie JavaScript file with practical utility functions for use in web pages. |
labenv/src/net/sf/basedb/labenv/LabEnv.java | Small class with static methods for conversion and comparison of values. |
labenv/resources/reports/boxplot.js | Copy of Reggie JavaScript file with practical utility functions for creating box plot statistics. |
comment:9 by , 11 years ago
Notes on configuration file labenv-config.xml
:
Configuration file labenv-config.xml
contains the settings for the lab environment measurement and storage. The first version has the following structure:
<?xml version="1.0" encoding="UTF-8"?> <parameters> <title>...</title> <!-- <dbfilename>labenv.db</dbfilename> <timeoutinseconds>5</timeoutinseconds> <storageintervalinseconds>300</storageintervalinseconds> <numberofmeasurementsperstoredvalue>5</numberofmeasurementsperstoredvalue> --> <dbfilename>...</dbfilename> <timeoutinseconds>...</timeoutinseconds> <storageintervalinseconds>...</storageintervalinseconds> <numberofmeasurementsperstoredvalue>...</numberofmeasurementsperstoredvalue> <labsensors> <labsensor> <number>...</number> <url>...</url> <name>...</name> <startdate>...</startdate> <alarms> <alarm> <starttime>...</starttime> <endtime>...</endtime> <temperaturemin>...</temperaturemin> <temperaturemax>...</temperaturemax> <humiditymin>...</humiditymin> <humiditymax>...</humiditymax> <userlist> <user>...</user> </userlist> </alarm> </alarms> </labsensor> ... </labsensors> </parameters>
Global configuration values:
XML tag name | Description |
timeoutinseconds | Timeout in seconds when trying to connect to lab sensors. |
dbfilename | Name of the database file used to store lab environment data. |
storageintervalinseconds | Time interval in seconds between storage of measurement values for lab sensors. |
numberofmeasurementsperstoredvalue | Number of measurements performed to get an average value to store. The measurements are performed at times evenly distributed over the time interval between storage of values, e.g. if data values are stored every 300 seconds (5 minutes), and the number of measurements per stored value is 5, a measurement is performed every 300/5 = 60 seconds. The time stored together with a stored data value is set to the mean of the times for the measurements used to calculate the average stored data value. |
Configuration values for each lab sensor:
XML tag name | Description |
number | Number used to identify the sensor in the lab environment database. It is simplest to think of the sensor number as identifying the location, the sensor is placed in. |
url | URL used to access the lab sensor (configured in the internal web server in the physical sensor). |
name | Name of the sensor/room the sensor is located in (configured in the internal web server in the physical sensor as the title of the generated web page). This name is used to identify sensors in the sensor selection menu in web pages in the Lab environment BASE extension package. |
startdate | A date indicating when the sensor was put in use, and is currently not stored in the database. A sensor with a blank start date will be skipped when measuring and storing lab environment values. This can be used to avoid long connection times before time-out for sensors, that are disconnected from the network during longer time periods. |
alarms | An optional list of alarm tags, defining conditions for desired alarms. If triggered, an alarm will be sent to the users in the user list for the alarm in question. (Method of sending alarm is currently not decided.) |
An example with two lab sensors is as follows:
<?xml version="1.0" encoding="UTF-8"?> <parameters> <title>LabEnv options&hellip;</title> <!-- <dbfilename>labenv.db</dbfilename> <timeoutinseconds>5</timeoutinseconds> <storageintervalinseconds>300</storageintervalinseconds> <numberofmeasurementsperstoredvalue>5</numberofmeasurementsperstoredvalue> --> <dbfilename>labenv.db</dbfilename> <timeoutinseconds>5</timeoutinseconds> <storageintervalinseconds>300</storageintervalinseconds> <numberofmeasurementsperstoredvalue>5</numberofmeasurementsperstoredvalue> <labsensors> <labsensor> <number>1</number> <url>http://sensor1.onk.lu.se/</url> <name>S1 Instrument room (B32C2)</name> <startdate>2014-03-06</startdate> <alarms> </alarms> </labsensor> <labsensor> <number>2</number> <url>http://sensor2.onk.lu.se/</url> <name>S2 RNA lab (B33A2)</name> <startdate>2014-03-06</startdate> <alarms> </alarms> </labsensor> </labsensors> </parameters>
Measurements from URL's http://sensor1.onk.lu.se/
and http://sensor2.onk.lu.se/
will be stored every 5th minute (5*60 = 300 seconds) in database file "labenv.db
". The URL's and sensor names should correspond to the configuration of the internal web servers in the physical sensors. The URL's and names displayed in this example differ from those actually used for the sensors at the Department of Oncology at Lund University, as Swedish terminology is used locally.
- The important thing to note, is that the measurement data will be stored in the database for the sensor number defined in the configuration file. This lab sensor number is therefore not just a trivial index number for internal use in the XML configuration file. When sensors are added for new locations, they should normally be configured with a new unique number. However, if a malfunctioning sensor for an important location needs to be temporarily exchanged for a working sensor internally configured for a another, less important, location, and there is no time to re-configure the latter sensor internally, the lab environment configuration file can be modified so the measurements from the temporary sensor are stored for the sensor number of the malfunctioning one.
comment:10 by , 11 years ago
Component: | net.sf.basedb.reggie → net.sf.basedb.labenv |
---|---|
Milestone: | Reggie v2.x → LabEnv v1.0 |
Ticket Milestone changed to LabEnv v1.0 and component to net.sf.basedb.labenv.
comment:11 by , 11 years ago
(In [2303]) Refs #529. Initial check-in of source code for lab environment BASE extension. The package allows obtaining temperature and humidity values from a number of ThermoWorks Thermo Recoder TR-702W sensors at regular intervals, and storing the data in a special database. The stored data can be exported or directly inspected statistically. The first version allows inspection of the daily or weekly distribution of lab environment values.
comment:12 by , 11 years ago
comment:13 by , 11 years ago
comment:14 by , 11 years ago
(In [2307]) Refs #529. Class/file ReportTableUtil.java
in labenv/src/net/sf/basedb/labenv/util/
updated to hopefully fix problem with assigning an hour period string for times on dates when daylight saving time starts or ends (previous algorithm gave the number of hours after midnight):
- Public method
String getCurrentPeriod(Date currentDateTime, String viewType)
updated forviewType
equal to "dailyDistribution
" to use algorithm usingSimpleDateFormat(...)
, that hopefully will be correct also at start and end of daylight saving time.
comment:15 by , 11 years ago
(In [2308]) Refs #529. Class/file ReportTableUtil.java
in labenv/src/net/sf/basedb/labenv/util/
updated to reuse SimpleDateformat
object :
- New private instance variable
DateFormat hourPeriodDateFormat
added together with public accessor method, that creates an instance ofSimpleDateFormat("HH")
, ifhourPeriodDateFormat
isnull
. - Public method
String getCurrentPeriod(Date currentDateTime, String viewType)
updated forviewType
equal to "dailyDistribution
" to call new public methodgetHourPeroidDateFormat()
to get aDateFormat
object to use.
comment:16 by , 11 years ago
Design update:
The initial source code check-in in changeset [2303] turned out to have a number of flaws, that will be corrected in sequential updates:
- Service class
LabEnvService
should be moved fromsrc/net/sf/basedb/clients/web/extensions/service/
tosrc/net/sf/basedb/labenv/service/
, in order to be a sub-package tonet.sf.basedb.labenv
, without risk of interfering with outside classes. Configuration fileMETA-INF/extensions.xml
should be updated with the change. - Jar files
commons-httpclient-3.1.jar
andsqlite-jdbc-3.7.15-M1.jar
should be moved fromlib/compile/
toMETA-INF/lib/
, in order to be distributed with thelabenv.jar
file, as the library files in questions are not included with the BASE distribution. Ant build filebuild.xml
should be updated to include files inMETA-INF/lib/
in the class path when compiling. - Only JAR file
slf4j-api-1.6.4.jar
is needed in directorylib/compile/
, whileslf4j-log4j12-1.6.4.jar
could be removed. - Text files
LICENSE
,README
, andRELEASE
should be included in the distribution. - The configuration file
labenv-config.xml
included in the distribution should not include specific values, except for tag numbers for list tag items, needed to make the file valid. - The configuration routine should look for the configuration file
labenv-config.xml
directly, instead of first looking for the BASE configuration file, and then looking for the LabEnv configuration file in the same directory (the reason for the latter procedure was to be able to refer to the directory, the configuration should be located in, in the error message for a missing file). Also, since the functionality of the lab environment extension depends critically on the data in the configuration filelabenv-config.xml
, an exception should be thrown if no configuration file is found, and the extension should not start.
comment:17 by , 11 years ago
(In [2313]) Refs #529. Update of lab environment service class package:
- Service class
LabEnvService
moved fromsrc/net/sf/basedb/clients/web/extensions/service/
tosrc/net/sf/basedb/labenv/service/
, in order to be a sub-package tonet.sf.basedb.labenv
, without risk of interfering with outside classes. - Configuration file
META-INF/extensions.xml
updated with the change.
comment:18 by , 11 years ago
(In [2314]) Refs #529. Update ensuring that JAR-files not included with the BASE distribution, should be included in the LabEnv distribution:
- Jar files
commons-httpclient-3.1.jar
andsqlite-jdbc-3.7.15-M1.jar
moved fromlib/compile/
toMETA-INF/lib/
, in order to be distributed with thelabenv.jar
file, as the library files in questions are not included with the BASE distribution. - Text file
readme.txt
inlib/compile/
updated with the removal of filescommons-httpclient-3.1.jar
andsqlite-jdbc-3.7.15-M1.jar
from the directory. - Ant build file
build.xml
updated to include files inMETA-INF/lib/
in the class path when compiling.
comment:19 by , 11 years ago
(In [2315]) Refs #529. Update of JAR files included for compiling in directory lib/compile/
:
- JAR file
slf4j-log4j12-1.6.4.jar
removed from directorylib/compile/
, since only JAR fileslf4j-api-1.6.4.jar
is needed. - Text file
readme.txt
inlib/compile/
updated with the removal of fileslf4j-log4j12-1.6.4.jar
from the directory.
comment:20 by , 11 years ago
comment:21 by , 11 years ago
comment:22 by , 11 years ago
(In [2321]) Refs #529. Lab environment extension updated to look directly for configuration file labenv-config.xml
, and not start the service if it could not be found:
- Class/file
LabEnvironmentConfiguration.java
insrc/net/sf/basedb/labenv
updated in private methodvoid fetchConfiguration()
to look directly for configuration filelabenv-config.xml
, and to forward any exception thrown. - Class/file
LabEnvironment.java
insrc/net/sf/basedb/labenv
updated to forward any exception thrown by theLabEnvironmentConfiguration
instance created. - Class/file
LabEnvironmentServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated in private methodJSONObject createLabSensorConfigList(JSONObject json)
to forward any exception thrown by theLabEnvironment
instance recast as aServletException
. - Class/file
LabEnvironmentStatisticsServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated in private methodsJSONArray createLabEnvironmentDailyDistributionReport(JSONArray jsonStatisticsPlotArray, List<LabEnvironmentData> lthdList, Date startDate, Date endDate, String chartVariant, String viewType, String sensorName, String weekdayFilter)
andString fetchChartTitle(String viewType, String variableType, Date startDate, Date endDate, String weekdayFilter)
to forward any exception thrown by theLabEnvironment
instance recast as aServletException
. - Class/file
LabEnvironmentStorageUtil.java
insrc/net/sf/basedb/labenv/util/
updated in public constructorLabEnvironmentStorageUtil()
and public methodList<LabEnvironmentData> queryLabEnvDb(Integer sensorNumber, Date startTime, Date endTime)
to catch any exception thrown by theLabEnvironment
instance and print its content to standard output.
comment:23 by , 11 years ago
Design update:
- Update in changeset [2321] is unsatisfactory in two ways:
1. An exception thrown when trying to start the extension is not reported in the BASE window, only in the standard output.
2. The Tomcat server needs to be restarted, if one has changed the configuration filelabenv-config.xml
, and wants it to be reloaded; just restarting the extension is not enough.
Possible remedies:
- A
RuntimeException()
can be thrown by public methodvoid start()
in classLabEnvService
, without upsetting theServiceControllerAction
Interface. By also letting private methodvoid fetchConfiguration()
in classLabEnvironmentConfiguration
forward the contents of a caught exception as aRuntimeException
, one doesn't have to declare that a number of methods throw exceptions. - A new public method
void reloadLabEnvironmentConfiguration()
could be added to singleton classLabEnvironment
, for forcing a reload of the configuration file. ClassLabEnvService
could then be updated in public methodvoid start()
to callreloadLabEnvironmentConfiguration()
, to ensure that the configuration file is read, when the service is started.
comment:24 by , 10 years ago
(In [2326]) Refs #529. Lab environment extension updated to report in the BASE window, when configuration file labenv-config.xml
is missing, in which case the extension cannot start. Also, the configuration file is now reloaded every time the extension starts, even if the Tomcat server isn't restarted:
- Singleton class/file
LabEnvironment
updated with new new public methodvoid reloadLabEnvironmentConfiguration()
, for forcing a reload of the configuration file. - Class/file
LabEnvironmentConfiguration
updated in private methodvoid fetchConfiguration()
to forward the contents of a caught exception as aRuntimeException
, avoiding the need to declare that a number of methods in other classes throw exceptions. - Class/file
LabEnvService.java
insrc/net/sf/basedb/labenv/service/
updated in public methodvoid start()
to call new public methodvoid reloadLabEnvironmentConfiguration()
in classLabEnvironment
, and to forward the contents of a caught exception as aRuntimeException
, which doesn't violate theServiceControllerAction
Interface. - Class/file
LabEnvironmentServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated in private methodJSONObject createLabSensorConfigList(JSONObject json)
to no longer forward exceptions thrown by theLabEnvironment
instance recast as aServletException
. - Class/file
LabEnvironmentStatisticsServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated in private methodsJSONArray createLabEnvironmentDailyDistributionReport(JSONArray jsonStatisticsPlotArray, List<LabEnvironmentData> lthdList, Date startDate, Date endDate, String chartVariant, String viewType, String sensorName, String weekdayFilter)
andString fetchChartTitle(String viewType, String variableType, Date startDate, Date endDate, String weekdayFilter)
to no longer forward exceptions thrown by theLabEnvironment
instance recast as aServletException
. - Class/file
LabEnvironmentStorageUtil.java
insrc/net/sf/basedb/labenv/util/
updated in public constructorLabEnvironmentStorageUtil()
and public methodList<LabEnvironmentData> queryLabEnvDb(Integer sensorNumber, Date startTime, Date endTime)
to no longer catch exceptions thrown by theLabEnvironment
instance and print its content to standard output.
comment:25 by , 10 years ago
(In [2328]) Refs #529. Lab environment extension updated to increase similarity of logging with that used by BASE. Package org.apache.log4j.Logger
exchanged for org.slf4j.Logger
and org.slf4j.LoggerFactory
. Logger object properties changed from protected static
to private static final
. Affected files:
- JAR file
log4j-1.2.16.jar
removed fromlib/compile/
. - Class/file
LabEnvironment.java
insrc/net/sf/basedb/labenv/
updated. - Class/file
LabEnvironmentConfiguration.java
insrc/net/sf/basedb/labenv/
updated. - Class/file
LabEnvService.java
insrc/net/sf/basedb/labenv/service/
updated. - Class/file
LabEnvironmentServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated. - Class/file
LabEnvironmentStatisticsServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated. - Class/file
LabEnvironmentStorageUtil.java
insrc/net/sf/basedb/labenv/util/
updated. - Class/file
LabSensorUtil.java
insrc/net/sf/basedb/labenv/util/
updated.
comment:26 by , 10 years ago
(In [2329]) Refs #529. Lab environment extension updated in classes forwarding exception as a RuntimeException
to forward the full exception information, and not just the message:
- Class/file
LabEnvironmentConfiguration
insrc/net/sf/basedb/labenv/
updated in private methodvoid fetchConfiguration()
to forward the full information of a caught exception as aRuntimeException
. - Class/file
LabEnvService
insrc/net/sf/basedb/labenv/service/
updated in public methodvoid start()
to forward the full information of a caught exception as aRuntimeException
.
comment:27 by , 10 years ago
(In [2330]) Refs #529. Class/file LabEnvironmentConfiguration
in src/net/sf/basedb/labenv/
updated in private method void fetchConfiguration()
to throw a RuntimeException
with an appropriate message, if configuration file labenv-config.xml
could not be found. This should result in a better error message, if the configuration file is missing, the first time the extension is started.
comment:28 by , 10 years ago
comment:29 by , 10 years ago
comment:30 by , 10 years ago
(In [2337]) Refs #529. Lab environment extension updated to use default values where possible, if no configuration values are supplied in configuration file labenv-config.xml
, or a configuration value is out-of-bounds:
- Configuration file
labenv-config.xml
updated:
a. Database filename setting is put before settings for communicating with the sensors.
b. A comment block with default values for database filename, timeout in seconds, storage interval in seconds, and number of measurements per stored value is inserted before the configuration lines for these variables.
- Class/file
LabEnvironmentConfiguration.java
insrc/net/sf/basedb/labenv/
updated in private methodvoid fetchConfiguration()
to use default values for database filename, timeout in seconds, storage interval in seconds, and number of measurements per stored value, if no configuration values are supplied in configuration filelabenv-config.xml
, or a configuration value is out-of-bounds. Code for settings of configuration values moved to came in the same order as the values in the configuration file. - Class/file
LabEnvironmentStorageUtil.java
insrc/net/sf/basedb/labenv/util/
updated to not set a default database filename, since that is now handled by classLabEnvironmentConfiguration
.
comment:31 by , 10 years ago
comment:32 by , 10 years ago
comment:33 by , 10 years ago
(In [2342]) Refs #529. Lab environment extension updated by replacing all remaining System.out.println()
calls with logging using org.slf4j.Logger
and org.slf4j.LoggerFactory
(these calls are used in LabEnvService
to log that the service is started or stopped). Earlier calls to System.out.println()
that has been commented out are now removed. Also removal of some other code lines, that has been commented out. Affected files:
- Class/file
LabEnvironment.java
insrc/net/sf/basedb/labenv/
updated. - Class/file
LabEnvironmentConfiguration.java
insrc/net/sf/basedb/labenv/
updated. - Class/file
LabEnvService.java
insrc/net/sf/basedb/labenv/service/
updated. - Class/file
LabEnvironmentServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated. - Class/file
LabEnvironmentStatisticsServlet.java
insrc/net/sf/basedb/labenv/servlet/
updated. - Class/file
LabEnvironmentStorageUtil.java
insrc/net/sf/basedb/labenv/util/
updated. - Class/file
LabSensorUtil.java
insrc/net/sf/basedb/labenv/util/
updated.
comment:34 by , 10 years ago
Design discussion:
- Re-loading the
LabEnv
extension sometimes leads to ajava.lang.UnsatisfiedLinkError: org.sqlite.NativeDB._open(Ljava/lang/String;I)V
error, due to multiple copies of the sqlite JAR file. It is therefore recommended that thesqlite-jdbc-3.7.15-M1.jar
file is copied to the Tomcatlib
directory, instead of being distributed with the extension in itsMETA-INF/lib/
directory. JAR filesqlite-jdbc-3.7.15-M1.jar
can be downloaded from https://bitbucket.org/xerial/sqlite-jdbc/downloads/. If the error should still occur, the problem can normally be fixed by restarting Tomcat.
comment:35 by , 10 years ago
(In [2345]) Refs #529. Lab environment extension updated by moving JAR file sqlite-jdbc-3.7.15-M1.jar
back from META-INF/lib/
to lib/compile/
, in order to try to avoid a java.lang.UnsatisfiedLinkError: org.sqlite.NativeDB._open(Ljava/lang/String;I)V
error, due to multiple copies of the sqlite JAR file. JAR file sqlite-jdbc-3.7.15-M1.jar
therefore should be copied to the Tomcat lib
directory, instead of being distributed with the extension. (The JAR file can be downloaded from https://bitbucket.org/xerial/sqlite-jdbc/downloads/):
- JAR file
sqlite-jdbc-3.7.15-M1.jar
removed from directoryMETA-INF/lib/
. - File
MANIFEST.MF
inMETA-INF/
updated by removing filesqlite-jdbc-3.7.15-M1.jar
from the class path. - JAR file
sqlite-jdbc-3.7.15-M1.jar
added to directorylib/compile/
. - Text file
README
updated in the installation instructions to include directions on downloading JAR filesqlite-jdbc-3.7.15-M1.jar
from https://bitbucket.org/xerial/sqlite-jdbc/downloads/, and placing it in the Tomcat
lib/` directory.
- If the error should still occur, the problem can normally be fixed by restarting Tomcat.
comment:36 by , 10 years ago
(In [2346]) Refs #529. Lab environment extension updated in outermost Ant build file build.xml
for target "package
":
- Entries for files
labenv.jar
(via${jar.name}
),README
, andLICENSE*
removed fromtarfileset
for directory ".
", since these are already included in target "dist
", that target "package
" depends on. - A new
tarfileset
for directory "lib/compile
" added, in order to include jar filesqlite-jdbc*.jar
in thelabenv-*.tar.gz
package.
comment:37 by , 10 years ago
comment:38 by , 10 years ago
(In [2349]) Refs #529. Lab environment extension updated to only output 2 decimals when exporting temperature and humidity data:
- Data access object class/file
LabEnvironmentData
insrc/net/sf/basedb/labenv/dao/
updated in public methodString toString()
to only output 2 decimals for temperature and humidity data.
comment:39 by , 10 years ago
(In [2352]) Refs #529. Lab environment extension updated when exporting temperature and humidity data:
- Data access object class/file
LabEnvironmentData
insrc/net/sf/basedb/labenv/dao/
updated in public methodString toString()
for temperature and humidity data, with removal of unnecessary string concatenation.
comment:40 by , 10 years ago
comment:41 by , 10 years ago
comment:42 by , 10 years ago
comment:43 by , 10 years ago
Milestone: | LabEnv v1.1 → LabEnv v1.0 |
---|---|
Resolution: | → fixed |
Status: | new → closed |
Ticket closed as a first version of the lab environment extension has been created.
Ticket accepted.