List of Figures

Figure 1.1 The role of activity learning in the design of an intelligent agent.

Figure 2.1 Examples of individual group actions and activities that are found in common everyday life settings.

Figure 2.2 Relationship between state (S), action, and activity.

Figure 3.1 PIR infrared sensor packaging (left) and internal architecture (right).

Figure 3.2 The two components of a magnetic contact switch encased in plastic (left) and a magnetic contact switch acting as a door open/shut sensor (right).

Figure 3.3 Tactile pressure sensors positioned in a chair (left) and interpreted as a pressure map (right).

Figure 3.4 RFID tag positioned beneath visible object barcode.

Figure 3.5 Increased acceleration values can indicate a start or stop of a motion.

Figure 3.6 A smart phone with a three-axis accelerometer (left) and a three-axis gyroscope (right).

Figure 3.7 Floor plan and sensor layout for the first floor of a smart home. Sensor identifiers starting with "M" are infrared motion sensors, sensors starting with "D" are magnetic door closure sensors, and sensors starting with "T" are ambient temperature sensors. The hand soap object sensor is located near the sink and the additional object sensors are in the kitchen closet. All object sensors are indicated with stars. Additional sensors included in these datasets are A001 (gas usage sensor attached to burner), A002 and A003 (hot and cold water consumption sensors, respectively), and P001 (whole-home power usage meter).

Figure 3.8 Positioning of wearable accelerometers on an individual's dominant arm (left) and hip (right).

Figure 3.9 Time plot of sensor activity during the Hand Washing activity. The x axis shows the time of day when the sensor message was generated. The y axis shows the identifier (on the left) and functional location (on the right) for each sensor generating messages.

Figure 3.10 Time plot of normalized accelerometer readings during the Hand Washing activity. The x axis shows the time of day when the sensor message was generated. The y axis shows the identifier (on the left) and placement (on the right) for each sensor. There are three lines for each sensor, corresponding to the x, y, and z axes.

Figure 3.11 Time plot of sensor activity during the Sweeping activity. The x axis shows the time of day when the sensor message was generated. The y axis shows the identifier (on the left) and functional location (on the right) for each sensor generating messages.

Figure 3.12 Time plot of normalized accelerometer readings during the Sweeping activity. The x axis shows the time of day when the sensor message was generated. The y axis shows the identifier (on the left) and placement (on the right) for each sensor. There are three lines for each sensor, corresponding to the x, y, and z axes.

Figure 3.13 Heat map of discrete event sensor usage in a smart home (left, motion sensor locations represented by circles in the image) and motion history image of a sit-to-stand transition movement (right). In both images, the pixel intensities indicate activities occurring in the corresponding region of the space. In the left picture, darker circles indicate areas in which more time was spent. In the right picture, lighter regions indicate more recent movement occurring in the region.

Figure 3.14 Plot of acceleration (Ax, Ay, Az) and rotational velocity (Rx, Ry, Rz) values for HIP accelerometer from the Sweeping activity data sample.

Figure 3.15 Frequency content in the accelerometer and gyroscope signal for the sensor placed on the hip while performing a Sweeping activity.

Figure 4.1 Plot illustrating the distribution of the start time of the Eating activity from a synthetic dataset. The three solid curves represent the distribution for breakfast, lunch, and dinner. The dashed curve is the sum of these distributions.

Figure 4.2 The GMM_EM algorithm.

Figure 4.3 Graphical representation of an HMM. The shaded nodes represent the observed variables-the sensor events or the feature vectors from the sensor stream. The white nodes represent the hidden variables, which correspond to the underlying activity labels. Note the direction of the arrow indicating that the underlying hidden states (activities) generate the observations (sensor events).

Figure 4.4 A HMM used to recognize four distinct activities.

Figure 4.5 The Viterbi algorithm to generate the most likely sequence of activity labels from a sequence of sensor observations.

Figure 4.6 A sample decision tree for classifying a data point as Bed Toilet Transition, Personal Hygiene, Take Medicine, or Other.

Figure 4.7 ID3 decision tree algorithm.

Figure 4.8 Illustration of a SVM for a linearly separable classification problem.

Figure 4.9 Illustration of the soft-margin concept for SVM classification in the nonseparable scenario.

Figure 4.10 The two-class SVM algorithm.

Figure 4.11 Graphical representation of a linear chain CRF.

Figure 4.12 The Boosting algorithm to combine multiple classifiers.

Figure 4.13 The Bagging algorithm to combine classifiers.

Figure 4.14 The filter-based feature selection algorithm.

Figure 4.15 The PCA dimensionality reduction algorithm.

Figure 4.16 Plot of the data points projected onto a three-dimensional subspace using PCA. The axes in the figure represent the three principal components.

Figure 5.1 The AR process includes the stages of raw sensor data collection, sensor preprocessing and segmentation, feature extraction and selection, classifier training, and data classification.

Figure 5.2 Illustration of alternative approaches to processing sensor data streams for AR. Sensor event types and timings are depicted by the vertical lines, where the line type indicates the type, location, and value of the sensor message. The sensor windows are obtained using explicit segmentation, sliding window extraction with a fixed window time duration, or sliding window extraction with a fixed window length (number of sensor events).

Figure 5.3 Rule-based activity segmentation algorithm.

Figure 5.4 Illustration of time dependency in a sliding window of sensor events.

Figure 5.5 Effect of on weights.

Figure 5.6 Illustration of sensor dependency in a sliding window of sensor events.

Figure 5.7 Intensity-coded MI between every pair of motion sensors (right) in a smart home (left).

Figure 5.8 Online change point detection algorithm.

Figure 5.9 Hybrid bottom-up/sliding-window segmentation algorithm.

Figure 5.10 Example ROC curve (left) and PRC (right).

Figure 5.11 Event-based evaluation of Hand Washing AR. The vertical solid lines indicate the performance segment boundaries. The top row shows the ground truth labeling of hand washing or sweeping and the bottom row shows the labels generated by an AR algorithm. Each segment mismatch is labeled with the type of error that is represented, either an overfill (O), underfill (U), insertion (I), deletion (D), merge (M), or fragmenting (F).

Figure 6.1 Time spent by Americans as reported by the Bureau of Labor Statistics14.

Figure 6.2 Example activity-attribute matrix. Each row represents a distinct activity and each column represents a separate high-level feature. The contents of the matrix indicate whether the feature is associated with the activity or not .

Figure 6.3 Frequency-based sequence mining algorithm.

Figure 6.4 Highest-value patterns found by frequency mining.

Figure 6.5 Example of the hierarchical discovery algorithm applied to an input sequence S. Each unique sensor event symbol is represented by a unique circle pattern in the diagram. A sequence pattern (P) is identified and used to compress the dataset into a smaller sequence . A new best pattern is found in the second iteration of the hierarchical discovery algorithm.

Figure 6.6 Hierarchical compression-based sequence mining algorithm.

Figure 6.7 Illustration of the topic model process. Each sensor event sequence is a mixture of topics (high-level features), each topic is a distribution over low-level sensor features, each sensor feature is drawn from one of the topics.

Figure 7.1 LZ78 string parsing algorithm.

Figure 7.2 The trie formed by the LZ78 parsing of the location sequence . The numbers in parentheses indicate the frequency of the corresponding phrase encountered within the sequence as it is parsed.

Figure 7.3 ALZ string parsing algorithm.

Figure 7.4 The trie formed by the ALZ parsing of the location sequence aaababbbbbaabccddcbaaaa.

Figure 7.5 Forecasted time elapsed values for Enter Home activity.

Figure 7.6 A regression tree to output expected number of seconds until the next occurrence of a Bed Toilet Transition activity. Each internal node queries the value of a particular variable and each node contains a multivariate linear regression function. Each of the regression functions generates a prediction value as a function of the lags and other input feature values. The features include lag variables, or values of the predicted variable at previous time units. Lag1...