activityhacker

I purchased an Apple Watch specifically to test out the activity tracking features. I didn’t care that the cheaper sport models were all sold out (I wound up with the 42mm stainless steel), because I planned to return it after playing around with it for several days. Having done so, I am sticking with the return plan.

Apple’s “Modular” Watch Face w/ Fitness Metrics Displayed

I was never bullish on the Apple Watch to begin with because of the 12hr battery life. While I’ve found that it usually gets through the day, the watch simply doesn’t provide enough utility to justify charging a second device every night. This is exacerbated when traveling, where the inductive puck with ~6ft cable can be cumbersome, in addition to the need to pack a second AC to USB adapter if you want to charge it alongside your phone. I think the max recharge rate for a mainstream consumer wearable is once per week, and even then, all my friend’s Pebbles and FitBits have ended up dead in a drawer somewhere. If my FitBit Flex didn’t reliably last for 7 – 10 days, I’d probably have dumped it too.

The only activity feature of the Apple Watch that really jumped out at me is that it displays just the calories burned through motion, unlike FitBit products which also lump in the calories your baseline metabolic rate consumes. I cannot understand why FitBit insists on bundling baseline calories into daily expenditures, which makes it impossible to see the calories you’ve burned purely due to activity. Unless they’re concerned that users might drop dead before the end of the day – in which case, fitness trackers are the least of their worries – the app should show you only the calories burned due to motion, and then add in a days worth of baseline calories latter on. At the very least, it should break out calories due to movement separately, so that I don’t have to back-out my baseline rate, multiply that by the number of hours since midnight, and then subtract that product from what the device is showing me just so I can determine if I still need to go to the gym after that long walk to work. The UX team at FitBit should take note of Apple’s common sense approach here.

Activity Ring (left), where scrolling down reveals Activity Metrics (right)

I’m not a big fan of Apple’s visualization for activity, particularly because I don’t care about the metric represented by the innermost blue ring – the number of hours in which you stand for at least 1 minute. By default, the Apple Watch comes with the reminder to stand turned ON, which notifies the user if they have yet to stand up in the last 50 min. Even assuming that this practice provides real health benefits (I’m skeptical), I can’t imagine that anyone could endure hourly notifications for more than a day or two. Like many broader Apple Watch features (Lookin’ at you, creepy animated emojis), this seems like an idea that looked good on paper but flops in reality.

Movement / Calories Ring (left), where scrolling down reveals a bar chart of calories burned over time (right)

Swiping left from the Activity rings brings you into separate screens for calories, minutes of exercise, and hours in which you stood. Scrolling down from any of these three screens shows a bar graph over time, as seen above. Pretty, but there’s no innovation here.

I did want to compare the calorie expenditure calculated by the Apple Watch and my FitBit Flex, which I do in the spreadsheet below. All height and weight inputs were the same.

Comparison of Calories Calculated by FitBit Flex versus Apple Watch

Compensating for the underlying metabolic rate built into the FitBit Calorie expenditures ( I found mine to be 83 cal / hr by taking the device off and leaving it on a desk for an hour), I found that for a variety of activities the FitBit showed 10 – 20% higher calorie expenditures. This could be due to the fact that the Apple Watch has a built in heart rate sensor, and my resting heart rate is very low at ~48 bpm. So if the FitBit Flex (which lacks a heart rate sensor) has algorithms which assume a heart rate along the lines of the population average ~75 bpm, the Apple Watch’s measurement of a much lower HR value during resting periods could bring its calculations down versus the Flex. This is supported by the fact that the difference is the smallest (11%) during the activity in which my heart rate is highest (running).

In conclusion, I was underwhelmed by the Apple Watch’s activity tracking features. I didn’t find any really innovative features, and some, like the dashboard in the Activity app, were classic apple form over function:

Apple’s Activity App Dashboard

Sure, a month’s worth of concentric circles might look cool, but it can’t possibly be better at showing data over time than a simple bar chart like the FitBit app has. But maybe this is what consumers want…. I wouldn’t know, I’m an engineer, not a marketer. Finally, I don’t think the physical packaging of the device makes sense for very active people. It’s quite thick, which means that it extends off the wrist and bumps into things and scratches. I don’t mind when my FitBit gets banged up, as  can always get a new band for $8 on AMZN. But when I scratching the front of my $350 smart watch, and then have to look through those scratches to access apps, etc on a daily basis, I doubt I’m going to be wearing it anywhere active.

I just saw this new project pop-up on Kickstarter this morning, and am really hoping it’s not an April fools joke…. This device will do everything that my home brew tracker can, but with increased form factor flexibility and Bluetooth LE. A product like this represents a major step forward in the commoditization of wearable hardware, and dramatically lowers the barrier to entry for new applications so that developers and entrepreneurs can focus on what matters… the data! I wish MetaWear had been around when I started my blog, and I look forward to playing around with the first generation of hardware. Thank you to the MbientLab team for working to make open source activity tracking hardware a reality!

MetaWear: Production Ready Wearables in 30 Min or Less

I’ve had an iPhone 5s since late September 2013, and wanted to examine the relationship between the daily step data it records and the daily step data my FitBit Flex records. Of course, the data from the M7 should consistently underestimate my step count, since I’m always wearing my Flex but don’t always carry my phone with me (especially when I’m working out). This post is thus fundamentally different from previous posts, because the underlying variable is really about my personal phone-carrying habits (I’ve already done a brief experiment which incorporates a step-for-step comparison of the M7 and the Flex). Nevertheless, I wanted to see how accurate the M7 data would be in predicting my daily FitBit step count.

The iPhone 5s will only hold up to 7 days worth of data from the M7, so if you fail to launch a 3rd party app which downloads and stores the M7’s data on a weekly basis, you’ll start to loose steps. I was pretty good about checking-in with the Argus app in Sep, Oct, Nov and Dec, but started to get a bit spotty in Jan and Feb. Since 9/24/13, there were 116 days for which I had daily step counts from the M7 via Argus, and the data are plotted below:

M7 vs FitBit Flex

As expected, there is a pretty noticeable correlation, but before quantifying it I wanted to remove the outliers. I figured it was easiest to just manually remove data points for which either the M7 or the Flex recorded zero steps. The former could have been caused by my leaving my phone at home (which happened occasionally), and the latter was likely due to the periods when I wasn’t wearing the FitBit due to loosing one in the Bahamas or having my Force short out on me. Here is the plot of this data, along with a linear regression and R^2 coefficient:

M7 vs FitBit Flex Where Data Points with an X or Y Value =0 Have Been Removed

Having removed the zeros, I also decided to remove other data points which were also clearly outliers (shown in yellow above). Not only did these points not fit the generally trend, but the dates on which they occurred were highly suspect : several of the points occurred when I was changing time zones / crossing international date lines, while others occurred on days before / after I lost a Fitbit. I figured these confounding factors warranted their removal, and their absence generated the the plot below which had a considerably improved r^2 coefficient:

M7 Data vs FitBit Data with All Outliers Removed

There are several incredible features on this plot. The first is that the Y intercept of the linear regression (5257.7 Steps) is very close to the number of steps that my typical cardio workout routine (30 min on the elliptical) generates. I haven’t gathered data to support this, but I know qualitatively that I can always expect to gain about 5,000 steps on my FitBit after a 30 min workout. The M7, by contrast, gains no steps during this time as it is just sitting on the display stand streaming terrible Jason Statham movies off NetFlix. While this is seemingly a good explanation for the Y intercept, it is weakened by the fact that 1) I certainly don’t do cardio every day and 2) there must be an inherent baseline of daily of steps that my FitBit sees which my phone does not (for instance, getting dressed in the morning, showering, etc). It would be interesting, particularly for Apple or an app maker utilizing M7 data, to see what the average American’s baseline steps without their phone are, not including working out. This number could then be added back into M7 based step measurements in order to give a more accurate picture of a users real step count.

The second notable feature is that the slope of the line is almost exactly 1, which means that aside from any baseline + elliptical generated steps, the FitBit and M7 are almost spot on in their step counts. If the slope were considerably greater than or less than 1, it would be evidence that there is a systematic bias between the two measurements.

Finally, the R^2 value is decently high, which means that M7 data could probably be used to give me a rough idea of my FitBit step count on average. While it’s unlikely to be good enough to make determinations on a daily basis, if I can keep my phone on me for most of the day, I could probably bucket my weeks into high, med or low movement without having to keep track of going to the gym.

I wanted to test the algorithm from a previous post on multiple samples, so I recorded three trips on my tracker, Flex and M7 while walking around SF. The algorithm has two parameters than can be adjust in order to fine tune it: The cutoff point for the amplitude above and below 1G needed to trigger the peak sample counter, and the minimum number of data points above that threshold in order to record a peak (two peaks = one step). I sampled this data at 100Hz, because when I last used the algorithm I was running out of resolution.

I wrote a script MultiComplexCounter.m which took the vector sum for a given trip, and calculated the absolute percent difference between its calculated steps and the Felx’s steps for an array of different values for the two parameters mentioned above. I then took this data from the three trips and averaged it, to produce the table below.  Combinations of parameters which have less than a 3% error are in Red, less than a 10% error are in Yellow:

Averaged Percent Error For 3 Separate Trials Across and Spectrum of Values for Two Parameters

I was pretty stoked that the average percent error was so low in several places. I would imagine that the Flex’s error versus the actual number of steps walked is somewhere in this range, so I am not that far off. I think it is also comforting to see that there is a clear band which shows a relationship between the number of samples in the peak and the threshold of peak detection. As the threshold for beginning to count samples expands on either side of 1G, it makes sense that there would be fewer samples in that peak to be counted.

The data gets even better, however, when you look at the individual trials, which boast combinations of parameters which yield percent errors as low as 0.2%, 0.3% and 0.9%! To put this in perspective, a 0.2% difference from the Flex is 1 step in 500. That is surely just as accurate as anything FitBit can do! You can see all the data tables here in Trial Tables.

Analyzing the data further, if you plot all the parameter combinations which give errors of less than 1% (there is at least one combination for each trial) on an individualized basis, you can see that the black boxes representing them are all fairly tightly clustered:

Black Boxes for Individual Values less than 1% On Top of Averaged Data

If you then take the average values for the parameters the black boxes represent, and round as needed since the number of samples must be an integer, you find that the ideal values moving forward are a +/-Threshold = 0.1583 and an Average Samples per Peak of 7. When using this algorithm to analyze data moving forward, I will be sure to use these values!

Over the holidays, I started thinking that an activity tracker algorithm should be position agnostic, i.e. it shouldn’t make a difference whether you have your arm by your side, over your head, etc. It also shouldn’t matter what the loading on that arm is, and the algorithm should produce the same result whether you’re carrying a dumbbell back to the rack at the gym or simply walking down the street. It is the vibration caused by taking a step, which is transmitted through the body to wherever the tracker is being worn, that is really the key signal. I decided to investigate this.

I picked a nice level spot of sidewalk in SF where I walked 50 steps six times in a row, taking care to make the exact same strides each time by stepping on the regularly spaced expansion joints in the cement. Each time I made the trip, I held my left arm outstretched in one of the following positions:

• Down at my side, perpendicular to the ground
• Above my head, perpendicular to the ground
• In front, parallel to the ground and perpendicular to the plane of my chest
• Behind, parallel to the ground and perpendicular to the plane of my chest (or as close as I could get it)
• Left of me, parallel to the ground and the plane of my chest
• Right, parallel to the ground and the plane of my chest

In case that language isn’t clear, here’s an amazing diagram which shows the the single letter references I’ll use throughout this post to refer to each position:

DaVinci Caliber Sketch of the Six Different Positions

I also decided I would have a better chance of holding my arm steady if it was weighted, so I choose to hold a 5-pack of Coors Light in my left hand, grabbing onto the empty loop where the missing 6th beer should have been. This put a mass of around 1.8Kg at the tip of my arm,  but what I failed to anticipate until I was half way through the experiment was that the 5-pack oscillated back and forth on its plastic collar, potentially introducing undesirable signals. As I have often thought on Saturday and Sunday mornings, Coors Light turned out to be a bad decision. In future experiments, I will choose something less dynamic like a barbell or a book.

As I was pacing off my 50 steps with my arm in each position, I also recorded data from my Flex (located on the same wrist as my tracker) and my iPhone 5s M7 processor (in my left pants pocket). Here are the results:

Device Step Counts

Of course, the M7 was unaffected by the arm position because it was in my pocket, but both of these devices turned out to be pretty darn accurate in this setting. FULL DISCLOSURE: I lost my old Flex diving on a reef off the cost of Great Abco Island, so the measurements here are coming from a new device versus previous posts. Given the volume at which FitBit is manufacturing , however, I will assume that any variations  between the two devices is negligible.

The Raw Data for the 6 Position Trial is plotted below, along with the vector sum of the three axes. I’ve also labeled when the trials are occurring, to help them stand out from some of the other noise that’s in here (like me walking back to the other end of the sidewalk in order to repeat the exact same 50 steps):

Plot of All Six 50 Step Trials

I feel pretty good about all these, except for the trial B when the 5-pack was behind my back… This was a really awkward position to maintain, and combined with the oscillating beers makes it a pretty junk signal. I decided to throw it out, and plotted the vector sums for all the other 5 trials bellow:

Vector Sums for the 5 Good Trials

The next thing I wanted to do was revisit the methodology from an earlier post, and figure out what the average step looked like for the vector sum each of the 5 trials with the tracker in different orientations. Here are the results:

The “Average Step” From each of the 5 Trials

From this plot, we can learn that no matter what orientation the arm is in, a step looks pretty much the same when looking at the vector sum of all three axes. I then added the average of all 5 trials to the plot, and threw in some annotations which I believe will be useful in developing a new step counting algorithm:

Annotated Average Signals

As the plot shows, the feature of all these signals that is the most similar is the period of the step and the slope of the signal between 0.9Gs < x < 1.1Gs (see yellow circles). I therefor believe it will be interesting to build an algorithm with the following criteria for what constitutes a step:

1. Go from 0.9Gs to 1.1Gs in 1, 2 or 3 samples without any decrease in force (i.e. S1 < S2 < S3)
2. Remain above 1.1Gs for at least 5 samples
3. Go from 1.1Gs to 0.9Gs in 1, 2 or 3 samples without any increase in force (i.e. S1>S2>S3)
4. Remain below 0.9Gs for at least 5 samples

If all of the above criteria are met, a step would then be recorded. Obviously this is a continuous cycle, where you could start counting at either positions 1 or 3. I wrote a script ComplexCounter.m which approximated the above, although I wound up not coding in anything related to the slope because I found I often had no data points in that range due to a low sample rate. I adjusted the algorithm so that in order to count as a step, the signal had to remain above 1.1Gs or below 0.9 Gs for two continuous samples, and generated these results:

ComplexCounter.m Results

The algorithm performed pretty well, even when back tested against a much nosier signal from a previous post. Overall, I think I came away thinking that the right way to approach this problem is through the vector sum of the forces, since the signals look pretty much the same no matter how your wrist / arm are oriented. I also think there is something to the approach I’ve taken of measuring a minimum number of samples above a threshold, but I don’t believe my algorithm is sufficiently robust at present. I need to gather some more real world data to back test it with and potentially make some further tweaks.

I was excited to receive my Fitbit Force in the mail last week, because I’m constantly looking at my Flex to see what time it is (there is, of course, no clock on the Flex, so this habitual action frequently leaves me feeling quite stupid). The Force is definitely a bit heavier and almost twice as wide, but the quality of the display was excellent and the ability to interact with the device with a button rather than crude tapping gestures was great. Unfortunately, the Force is not even remotely waterproof! I wore it accidentally in the shower for 20 seconds before remembering to take it off, and the device shorted out. For about half a day, the Force flickered on and off while intermittently displaying the Fitbit logo, and condensed water was visible inside the display. Although it ultimately started working again, the problem reemerged after 5 minutes of washing dishes so I stopped wearing the device.

Condensed Water Inside the Force

Given the durability of the Flex, I was shocked that a device which feels just as rugged can’t even handle a splash of water. Having a display to show time and step count is a great feature, but if you can’t even wash dishes with it, I think there’s a brand-damaging design flaw in this product. The whole point of the Flex is that it’s thoughtless, and simply disappears into your life for days at a time until it needs to be charged. If getting my activity tracker wet is instead tantamount to spilling water on a mogwai,  that completely changes the use case and makes it stick out like a sore thumb in my daily routine. I’ll be returning Fitbit’s latest creation, and leaving “the force” to Starwars…

While reviewing some previous posts, it occurred to me that my methodology for identifying cross correlation sample signals – i.e.,  choosing a “representative” signal at an arbitrary point in time for an arbitrary duration – was not very scientific. Using any discrete sample walking signal (rather than the average walking signal) might result in lower than expected correlations. So, I decided to build a script which would automatically extract an average sample signal from a specified window of data, in the hope that it could be used in the future to generate better correlations.

I used the data from a previous post, and found a window of data that I consider to be representative of general walking, from time 160 < t < 180:

Plotted Data from All 3 Axis from Time 160 < t < 180

There are a lot of interesting features in the plot above which could be used to identify a step – for example, each Z axis peak seems to be accompanied by a Y axis peak, and every other local minima in the X axis seems to correspond to a peak in both the Y and Z axes. I will ignore these for now, and focus on the X axis, where it is clear that period of the signal is two peaks long, i.e. there is a larger peak, a smaller peak, and then the signal repeats. This makes sense in the real world, in which the force the logger experiences in the X axis is larger from the step of one foot (presumably, the step which occurs on the same side of the body on which the logger is being worn) than in the other. I created a script, signalextractor.m, which takes the data from an input window, runs a peak detection, and then extracts the data in two-peak long periods. Plotted on top of each other, the data for each two-step period looks like this:

Data from Multiple Steps in the X Axis

I was very pleased with the consistency between the various periods shown in the plot, although the discontinuity between the beginning and ending values is disconcerting. I will assume however, that this is simply due to the low sample rate, and that if I had sampled at the logger’s maximum rate of 200Hz, the fist and last values for each plotted period would be nearly the same.  I then used this extracted data to produce an average sample signal, which is plotted below ontop of the underlying sample data points:

Average Signal ontop of Multiple X-Axis Two-step Cycles

I now have a useful script which I can use in the future to generate an average sample signal, as well as a means for investigating the standard deviation of accelerometer readings at each point in the step cycle. It might be useful in a future step counting algorithm, for example, to reject any signal as a step if it falls outside of a given range at a given point in time.

Many thanks to TKM for letting me know about this DIY implant a bio-hacker recently embedded into his arm. As he aptly pointed out, I will never be this hardcore, although I’d like to think I would have done a better job with those sutures…

I wanted to use the same methodology from a previous post (Coffee Run) to analyze a new set of data,  so I took a  different route  around the block to buy a six pack of Coors Light at the corner store. I also wanted to test out my iPhone 5s’ M7 motion co-processor, so I kept my phone in my left pant’s pocket and used the Argus app to note the number of steps (any app should give the same results, since it’s just calling an API from the M7).  I used the correlationwindower.m script from an earlier post on the raw data (beerrundata), and generated the following result using the first block of steps just before t=100 as the sample data:

Plot of the Correlation of a Sample Signal (91<t<97) With the Dataset (blue) ontop of the Raw Data (Green)

Immediately, you can tell something is wrong because from 260 < t < 310 on the X axis, when I am clearly standing in line not moving, the correlation is really high. This led me to think about micro-oscillations that might be showing up in the noise during periods of non-motion, so I wanted to look at the data in frequency rather than time. I built a simple script to perform a discrete Fourier transform on the data using the fft command in Matlab (quickdft.m), and produced the following plot:

Fourier Transform of the Data from Each Axis

I was pretty excited by this data. First, the data from the three axes are all very consistent with each other, which gave me a lot of confidence that I can ultimately build an algorithm that can extract motion from any axis (versus just the X axis, currently). Secondly, I thought it was very interesting that the peaks for the Y and Z axis were greatest around 1 Hz, while the X axis peak was greatest at around 2 Hz. I timed my gate, and found that each step takes about a half second, so what must be going on is that the X axis is oscillating with each step, while the Y and Z axes are oscillating with each stepping cycle (i.e. both the left and right foot). This makes sense, given that my hand was positioned as follows:

Logger Positioning During Normal Walking Motion

So, you get an acceleration in the X axis with every step as you bob up and down while walking, you probably get some lurching back and forth in the Y axis which occurs with each step pair, and in the Z axis you sway side to side a bit with each step pair since no two legs are exactly the same length. It is also interesting to note the 2Hz frequency component of the Y axis signal, which is probably capturing some of that same individual step motion as the X axis because some part of my the raising and lowering with each step is actually occurring in the Y axis. At the same time, the absence of any 2Hz signal in the Z axis is very comforting as it reinforces the idea that the Z axis is experiencing a side to side sway.

I thought that perhaps creating a band pass filter might solve the problem of the phantom correlation data during periods of no motion; perhaps there was a high frequency noise signal with lower frequency elements that were seeping into the data. So, I created the following filter in Matlab’s filter design tool, which was launched with the fdatool command.

A Simple Bandpass Filter

I used the bandpass filter on the X axis raw data, and produced the plot below:

X Axis Data After Bandpass Filtering

Running a simple peak detection on this data  ([x_pks, x_locs] = findpeaks(xf,’MINPEAKHEIGHT’,.1,’MINPEAKDISTANCE’,10);) counted 549 steps, which is pretty close to the 539 recorded by my Fitbit Flex and the 536 recorded by the iPhone 5s. I then changed the parameters of the filter to focus even more tightly on the 2Hz frequency with the following settings (fstop 1.5, fpass1 = 1.8, fpass2 = 2.2, fstop2 = 2.5), and counted only 507 steps, which led me to believe that the previous answer was mostly just lucky, although it is possible that tightening the filter might have excluded steps which occurred at a slower pace. Overall, I was a little disappointed with this experiment in frequency, but I learned a lot which can hopefully be applied to future experiments.

Found this really interesting blog post (pdf here: Fitbit 11-23-10) from Fitbit’s chief data scientist explaining how steps do not necessarily equate to calories. Will have to check out the two authors mentioned in the post as well….  Of course, I need to figure out how to reliable detect steps first! It had not occurred to me to look for academic research on the the matter, however, so perhaps I should look for some literature on the subject before blinding performing my next analysis.