Example application for accelerometer/gyroscope processing on Android

21 replies [Last post]
paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
+1
0

In the previous parts we have seen how to simulate the gravity vector with the gyroscope if the accelerometer is not able to measure gravity correctly (because it is subject to motion acceleration too beside the gravity acceleration) and how to update this simulated gravity vector from the accelerometer to prevent the accumulation of measurement errors. At the end of the previous post we had a gravity vector which was normally taken from the accelerometer but our algorithm switched to gyroscope simulation automatically when needed.

We need just one more step to enable cool applications and that step is very evident. If you remember my Droidcon 2011 presentation (slide 24), the acceleration measured by the accelerometer is the sum of gravity and motion acceleration. If both are present, they cannot be separated in the general case without additional sensor input. But now we have the gravity acceleration which is reasonably exact even if the accelerometer is subject to motion acceleration too. This means that we can extract the motion acceleration by subtracting the simulated gravity acceleration from the acceleration measured by the accelerometer.

This opens the way for applications based on dynamic movements much like the Wii games with Motion Plus accessory (which actually do the same as the Motion Plus accessory is really just a 2+1 axis gyroscope).

The example program is attached to this post. You have to be logged in to the Sfonge site to access it.

After all these simulation script, let's see something hands-on. The example program is the implementation of all these ideas we have seen so far. You need an Android phone with gyroscope to test it. The gyroscope processing algorithm is implemented in SamplingService.java. GyroAccelActivity is really doing only visualization, getting callbacks from SamplingService.

When started, the service enters the calibration state. This state is necessary to get a good fix of the initial gravity vector so the device should not be moved when calibrating. The initial gyroscope drift experienced on my Nexus S also ceases during the calibration period.

Then the service enters the measuring state and implements everything we discussed so far. It samples the accelerometer and the gyroscope paralelly, simulates the gravity vector if it finds out that the accelerometer is probably subject to motion acceleration and calls back the activity with the calculated motion acceleration vector. The acceleration is shown as movement of a large white ball (quite ugly, actually). The x and the y components of the motion acceleration move the ball horizontally and vertically while the size of the ball is calculated from the z component of the motion acceleration.


Note that the motion acceleration is expressed in the coordinate system of the device itself, so you may get some unexpected result if the device is not parallel to the floor surface. As we have the gravity vector, rotating the motion acceleration vector to the Earth's coordinate system is simple exercise (took me several days to get it right ;-)). We calculate two rotations (around z and y axes) to rotate the gravity vector into the z axis. At the same time we rotate the motion acceleration vector with the same rotations. This yields a normalized motion acceleration vector that would have been measured if the device were parallel to the surface.

If you are done with playing the ball, you will notice that analyzing these 3D acceleration vectors are not trivial. You can try setting the SamplingService.DEBUG field to true, then the application produces the usual /sdcard/capture.csv file (beware, it can be large) that you can download to your computer and visualize with the attached Sage script.

rajeshjnu's picture
Offline
Joined: 29 Aug 2013
Points: 6
Capturing Motion of Mobile with earth as reference

Sir,

I want to measure my mobile's movement with respect to earth's coordinates (North-X, East-Y, Down-Z)

how is it possible using the sensors (Accelerometer + Gyroscope)?

Please reply

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Measuring displacement

Measuring displacement (location change) is very hard using the accelerometer+gyroscope. The main difficulty is drift: accelerometer measures acceleration and you have to integrate twice to get displacement. Any error in the accelerometer output is magnified enormously by the double integral. If you have a reliable distance input (e.g. step counter) then you can navigate with the compass and the distance input for a while using dead reckoning.

Rurouni_Asuna's picture
Offline
Joined: 24 Feb 2014
Points: 6
measure distance

i've read your example, on SamplingService you send iGyroAccel.diff( v[IDX_X], v[IDX_Y], v[IDX_Z]); those are de new position to paint the ball, but finally this new vector represent?? acceleration on each axe? or just a point, finally how can i integrate or what should i integrate twice to get the distance on your example? (I hope you understand my English) thanks a lot.

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Distance

The diff() parameters represent acceleration expressed in the device's coordinate system. I warn you, however, that getting distance is minimally very hard, I would say impossible. In order to get distance from acceleration, you have to integrate acceleration twice. A constant error term in acceleration introduces a lineraly growing error in velocity and an error term growing by the square function in distance. The accelerometers in these devices are not very exact so be prepared for a massive error. Other consideration is that even if accelerometer readings were exact, implementing the integration step would not be easy, read the error discussion of the Euler method.
In short: if you want to measure distance, it is better to exploit a known property of the movement, e.g. step counting in case of walking. In case of vehicles, you can try to obtain velocity/displacement data from the vehicle (e.g. in case of a car).

chetan169's picture
Offline
Joined: 29 Aug 2013
Points: 6
how to FUSE Accelerometer and Gyroscope data

Hello sir,
your explaination is too good i loved it.
i have a doubt
i got accelerometer anf gyro readings i displayed it on LCD.
just wanted to know how to remove gravity from accelerometer reading i saw your explaination bt am not getting it.
Is there any physics related to it.
any formula. How to relate Gyroscope value with accelerometer value.
Sir sorry to trouble you....plzzzzz help...

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
This presentation may
sattinrsc's picture
Offline
Joined: 8 Jul 2013
Points: 6
More info on Accelerometer_gyro algorithm

hi Gabor,
you had provide good info on accelerometer_gyro data and it processing.
i am working on same sensor fusion from scratch and facing some problem. I am working on microcontroller based board using "c" coding.I have one evalution board ATAVRSBIN1 from ATMEL which have tri-axis gyro(itg3200) and accelerometer(BMA150). This board provide the accel and gyro data samples with timestamp in file(txt/csv) using GUI on PC through serial port.
I am not able use these samples to get accelerometer vector and update this with gyro data.
In your Post the provided algorithm coding file in Java since i have no experience on java. so if possible can you help me to provide the same analysis of acceleration vector and gravity vector in "C" or simple mathematical form, for better understanding and implementation as a beginner.

thank you in advance

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Hi, Satti, There is a

Hi, Satti,
There is a presentation that describes the idea in general.
http://www.slideshare.net/paller/better-motion-control-using-accelerometergyroscope-sensor-fusion
With regards to the algorithm, I propose that you take a look at the attached phone_gyro.py file which is a prototype implementation of the algorithm for a mathematical program called Sage (it is really a .py file, it is just Sfonge's forum that renames it to .txt). You can also take a look at the processCalibrating/processMeasuring methods in SamplingService.java, those are really understandable for non-Java programmers too.

fatjons's picture
Offline
Joined: 11 Feb 2012
Points: 7
sensor and gps loging on sqlite

I'm very interested for a project of mine to log sensor data on a sqlite file, but cannot achieve this goal so far i can only write the sensor buffer to file but this method is prone to buffer overflow.
I'm using the android phone as a tool to log sensor and gps data for a long time, let say 2 hour and than get this data for further study.
SQLITE is a good choise because theoretically supports high frequency data feed which is my case.
Please let me know if there is any solution i can use.

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
SQLite

I am not sure I understand your problem. You say that you can't write sensor data to a file because file access is not fast enough? I have no such experience and SQLite definitely cannot be faster than file access as SQLite itself is based on file access.
If you only want to dump sensor data to persistent storage to be downloaded later, the best choice is a file. If you want to search and process the data on the phone, it may be necessary to use SQLite database. But speed is definitely not a factor here.

fatjons's picture
Offline
Joined: 11 Feb 2012
Points: 7
Wham i'm trying to say is

Wham i'm trying to say is that the only way i'm able to write the data to the file is through buffer dumps which depending on the frequency of the data, normal, game or fast, cause the buffer to overflow if i want to retrive data for a long period.
So it's obvious to try writing the data to sqlite as they happen, and this is what i cannot achieve.

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Dump frequency

Did you check SamplingService.java in the attached project? That class write data to file using simple PrintWriter. Why wouldn't that solution work for you?

stefy10's picture
Offline
Joined: 20 Mar 2012
Points: 7
sampling rate of android acceleromerter

Hi..
I want sampling frequency 20Hz,ie 20 samples/second.But my phone accelerometer has sampling frequency of only 17Hz.Is there any way to give delay by program to get 20 samples?

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Sampling frequency

Hi, Stefy,
As described in this presentation, sensor sampling frequency in Android is predefined and is not very stable. If the closest frequency to 20Hz your phone is able to produce is 17Hz, you cannot change that programmatically (at least not from application level).
There is one thing you can do: interpolation. By using the Whittaker-Shannon interpolation formula, you can calculate any point of the continuous-time analog signal from its samples (if you satisfy some bandwidth criteria that are normally satisfied with these sensors). With this formula you can basically resample your signal with any sampling rate. The same technique can be used to compensate for the uneven sensor sampling experienced in Android phones.
But before you do that, I would reconsider whether you really need 20 Hz exactly and not just 17 Hz.

stefy10's picture
Offline
Joined: 20 Mar 2012
Points: 7
FFT

Hi,
I have done FFT in phone by taking the accelerometer reading.But its frequency is changing randomly evenif I placed the phone simply on the table.Is this happening due to any error in sampling?How to correct it?

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
FFT

Remember the presentation I recommended to you: sampling rate in Android phones is has significant variation. Fortunately every sample comes with a timestamp. This allows you to resample the signal using the Whitakker-Shannon formula.
So my recommendation is:

  • Collect a buffer of accelerometer samples, along with their timestamps
  • Use the Whitakker-Shannon formula to produce another buffer of evenly sampled series with the sampling frequency of your choice (there are restrictions but resampling with 20Hz when the original sampling frequency was about 17Hz and the relevant signal is maybe 2-3 Hz does not cause any problems)
  • Apply FFT to the resampled signal
viet vo's picture
Offline
Joined: 9 Apr 2012
Points: 8
more information in AR

Because of popular papers related to AR problem, 25Hz on realtime smart phone ( X6, or android OS) is better, and the uncertainty of Hz caused from onSensorChange() android,
From my experiment, NORMAL mode is so bad, we can not extract better features when we apply filtering
From CPU and battery consumption of GAME mode take 40% CPU on Samsung galaxy note ( for only sensor collecting data application)
From your presentations, I would like to supply these things:
- Other advices such as: 25Hz is ok for AR,
- UI is acceptable choice for cost consumption or qualified data demand
normally, on samsung galaxy note 17Hz for UI mode, we can apply linear or cubic spline interpolation to increase to 25Hz. We should apply to all 3-axises at specific time (predefine algorithm to find this time, suggest: compare from variances of axises.
for filtering: smooth filtering is ok, but I refer to use wavelet db 4-5, since it can reconstructs original data more 90% accuracy.
almost each behavior of AR ( walking, running, take 0.8 -1 second)
from above reason, we should use wavelet db4 ( it takes 8 neighbours of each central point)

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Wavelet

Hi, Viet,
What do you mean by reconstructing signal with wavelets? Do you use wavelets for filtering like proposed in this paper? Or do you have something else in mind? Do you concentrate only on accelerator signal or gyroscope too?
Regarding interpolation: do you have experience of the noise introduced by linear or cubic interpolation compared to the theoretically perfect Whittaker-Shannon formula?
If you want to recognize walking, running, etc., with low processing requirements, you may be interested in this paper.

viet vo's picture
Offline
Joined: 9 Apr 2012
Points: 8
Daubichies wavelet

Hi Gabor,

- Reconstructing: wavelet in my case is only used to filter noise, by reconstruct we can check the accuracy of it.
- Wavelets for filtering: I use daubichies wavelet different from Morlet wavelet in your paper. I haven't applied Morlet yet.
Almost paper use db wavelet in gait authentication. From my experiment, accelerometer data consist many white noise (gaussian). Daubichies seperate our acceleration into 2 components (high pass, low pass).
For example with db8, it will manipulate correllation of ith signal to (i+15)th signal. With Fastest mode in HTC Nexus One ( from my experiment 22Hz), with window size 5 seconds.
We can get 110 samples, with db8 operation, we can maximum downsample 3 times ( level 3).
Regarding to this problem, there still has some papers use daubichies from level 4-8 for hard threshold (high frequency component is set to 0). But this has principal that more kept correlated sample derive accuracy is decrease. When low level is chosen , it will make noise.
Wherefore, db 5-6 is acceptable.
- I only use built-in acceleromter ( BMA 150) for widely popularity on smart phone.
- Interpolation: Linear interpolation is quick and easy but low precision, and new point is like step although it were used in many AR paper.
For smoother, cubic is considered since each interval uses 2 control points and 2 tangents.So, I only use linear and cubic in animation game and AR.
I will implement Whittaker-Shannon to compare.
- In optimization problem: until now, I only focus to simple features with portable trained model but also guarantee acceptable accuracy and resolving personalize model from known model and velocity problem with only accelerometer.

Thank you for your materials, it can be my further studies.

paller's picture
Offline
Joined: 11 Apr 2010
Points: 193
Material

Two further posts that may be relevant.

I will check out these Daubichies wavelets, I have no experience with them.
Personally, I believe less nowadays in frequency-domain analysis for movement recognition. I am more into pattern matching now, that's what the quoted paper does with PLL.

viet vo's picture
Offline
Joined: 9 Apr 2012
Points: 8
Thank

Thanks Gabor. I do these tasks in my master course.
After this period, I will study your material.