The IPIN 2026 onsite, online and offsite Indoor Localization Competition
IPIN Competition
The IPIN Indoor Localization Competition was born in 2014 to evaluate state-of-the-art localisation systems. During the competition, an actor walks along a given path while competing systems estimate the actor's position. The path is provided by competition organisers and is kept secret until the competition day.
The competition comes divided into Tracks with different flavours, namely onsite Tracks, online Tracks and offline Tracks.
Onsite Tracks have competitors reading sensors data coming in real time from a mobile device (smartphone, wearable or custom, depending on the Track). During the competition day the actor holds or wears the device while following the predefined path. Competitors can survey the area the day prior to the competition. During the competition, competitors estimate the actor's position from the device sensors' readouts. For some onsite Tracks competitors develop their own application for reading sensors (for example an app on a smartphone); for other Tracks, organisers read the sensors and send the readouts to a network host which is accessed by competitors who read them back in real time. In any case, competitors provide their position estimates in real time.
Online Tracks emulate the onsite Tracks scenario by means of a dedicated web application server called EvaalAPI. Well in advance of the competition, the Track organisers log data collected from a mobile device while following a given path and store it on the EvaalAPI. During the competition day, competitors alternately download a batch of sensor readouts from the EvaalAPI (typicall half a second worth of data) and send back a position estimate before downloading the next batch of data. In contrast with onsite Tracks, online Tracks are run on emulated real time, which is slower than real time (typically 3 times slower), can accommodate occasional delays (typically up to 30s) that may be caused by the competitor's system or by network problems, and can be replayed by the server for testing purposes.
Offline Tracks are run without any regard to data timing. The Track organisers collect the sensors readout in advance, and during the compeititon competitors download the whole data set of sensors readout in a single batch. Competitors then upload the resulting estimates all at once by a given deadline (typically 10 minutes).
The IPIN competition is meant to compare the performance of real algorithm and systems in a realistic setting. It is conceptually based on the EvAAL Framework, ensuring consistent and rigorous testing conditions.
The EvAAL Framework
The EvAAL Framework was conceived with the purpose of Evaluating AAL systems through competitive benchmarking. The idea is to gather together working systems, both prototypal and mature, and independently compare their performance in one or several specific areas, with the long- term objective of creating a set of evaluation benchmarks for indoor pervasive systems.
Core and extended criteria
EvAAL Framework's core criteria are:
- Natural movement of an actor: The agent testing a system moves spontaneously with a predefined script, including walking, resting or other actions.
- Realistic environment: The scene is a realistic setting.
- Realistic measurement resolution: The minimum time and space error considered are relative to the actor’s movement. The space resolution for a person is defined by the diameter of the body projection on the ground, which is set to 50 cm. The time resolution is defined by the time a person takes to walk a distance equal to the space resolution. In an indoor environment, considering a maximum speed of 1 m/s, the time resolution is 0.5 s.
- Third quartile of point Euclidean error: The accuracy score is based on the third quartile of the error, which is defined as the 2D Euclidean distance between the measurement points and the estimated points for single-floor settings.
EvAAl Framework's extended criteria additionally adopted by the first EvAAL competitions are the following:
- Secret path: The final path is disclosed immediately before the test starts, and only to the competitor whose system is under test. This prevents competitors from designing systems exploiting specific features of the path.
- Independent actor: The actor is an agent not trained to use the localization system.
- Independent logging system: The competitor system estimates the position twice per second, and sends the estimates to a logging application provided by the testing committee. This prevents any malicious actions from the competitors. The logging system is free software.
- Identical path and timing: The actor walks along the same identical path with the same identical timing for all competitors, within time and space errors within the above-defined resolutions.
EvAAL competitions were organized yearly during the lifespan of the universAAL FP7 project, from 2011 to 2013. In 2014, the International Conference on Indoor Positioning and Indoor Navigation (IPIN) organised the first IPIN competition, building on EvAAL's experience. Since then, the IPIN conference has hosted a competition based on the EvAAL framework, with both on-site and off-site Tracks.
Check che Current Competition on the menu to the top to know what's going on!
This is the new IPIN Competition web site! If you don't find all the information you need please notify us at info(at)evaal.aaloa.org and have a look at the old web site: https://evaal.aaloa.org/