Algorithm-assisted multi-day raw sensor data annotation
Human activity recognition using wearable accelerometers can enable in-situ detection of physical activities to support novel human-computer interfaces. Activity recognition algorithms use data on the motion and orientation of limbs to detect activities such as walking, sitting, and sleeping, among others. Many of the machine-learning-based algorithms require multi-person, multi-day, carefully-annotated training data with precisely marked start and end times of the activities of interest. To date, there is a dearth of usable tools that enable researchers to conveniently visualize and annotate multiple days of high-sampling-rate raw accelerometer data.
We developed “Signaligner Pro” – a data annotation tool to enable researchers to conveniently and quickly explore and annotate multi-day high-sampling rate raw sensor data with the assistance from state-of-the-art activity recognition algorithms. The tool visualizes high-sampling rate raw data and time-stamped annotations generated by existing activity recognition algorithms; the annotations can then be directly modified by the researchers to create their own, improved, labeled datasets.
Signaligner Pro visualizes raw sensor data (A), user added labels (B), and algorithm processed labels (C) to assist researchers in adding annotations of their choice to the raw sensor datasets. Signaligner has been currently deployed to work with raw accelerometer data from research grade devices such as Actigraph as well commercial smartwatches (with missing data).
Sensor data downsampling is performed by selecting a minimum and a maximum value in a given downsampling window. Then, we render the area between the minimum and maximum sensor reading using a mesh of triangles for a zoomed out (or low resolution) visualization. The labels in smaller window sizes (e.g., 1 second compared to several hours) are collapsed and expanded upon zooming in.
Aditya Ponnada, Seth Cooper, Qu Tang, Binod Thapa-Chhetry, Josh Miller, Dinesh John, and Stephen Intille. (2021, March). Signaligner Pro: A Tool to Explore and Annotate Multi-day Raw Accelerometer Data. In 2021 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, pp. 475-480, doi: 10.1109/PerComWorkshops51409.2021.9431110.
The software development was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (NIH) under award number UH2EB024407. The work was also supported by NU-TECH AWS credits award from Northeastern University, Boston, MA. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
This is a joint effort between Northeastern University’s mHealth Research Group, Crowdgames Lab, and Exercise Physiology Lab. We are also sincerely thankful to our teammates from sandboxnu.com for their support in improving this software as we continue to work with them.