The IPIN competition logo
The IPIN competition
Evaal logo

Technical Annex off-site

General description of the problem

A spectacular growth of indoor localization studies has been witnessed during the last decade, and the Wi-Fi Fingerprinting is the basis for many indoor localization approaches. This is mainly due to the proliferation of both wireless local area networks (WLANs or Wi-Fi’s) and mobile devices. Nowadays Wi-Fi’s can be found anywhere, and mobile phones have increasingly become an indispensable part of our daily lives and, therefore, we can assume that the user is placed where his/her mobile device is. The last generation of these devices (also known as smartphones) not only provides programmable abilities but they carry embedded sensors like GPS, accelerometer, gyroscope, microphone, camera, bluetooth, etc

Wi-Fi Fingerprinting is based on the Received Signal Strength Indicator (RSSI) of the visible Wireless Access Points (WAP’s). Commonly, two phases are needed: calibration and operation. In the calibration phase, a radio map of the area where the users should be detected is constructed. This phase consists on collecting Wi-Fi fingerprints at well-known locations. Later, during the operational phase, a user obtains the signal strength of all visible WAP’s that can be detected from his/her position and creates a location fingerprint. This fingerprint is compared to the training samples of the radio map. In the simplest solution, the user's location corresponds to the position of the most similar training sample.

One of the major advantages of Wi-Fi fingerprinting is that they do not require the installation of any additional hardware since they use the existing Wi-Fi infrastructure. Therefore, the location of the user can be obtained without additional infrastructures and costs. However, Wi-Fi was not natively designed to support a positioning function. Taking into account the existing obstacles introduced by the indoor environment (including reflections and multi path interference) the spread of radio signal in indoor environments is very hard to predict. In addition, the user typically carries the mobile device with him/her, being his/her motion or how the device is carried an important factor that affects the measured RSSI values.

In the competition, we propose to use the UJIIndoorLoc Database (https://archive.ics.uci.edu/ml/datasets/UJIIndoorLoc) to compare indoor location methodologies. UJIIndoorLoc is a multi-building and multi-floor database that is publicly available trough the UCI Machine Learning Repository.

Description of the dataset to be used

The UJIIndoorLoc database is already publicly available and it contains 21049 records: 19938 for training and 1111 for validation/test. Each record is directly related to a single capture and it contains the following 529 numeric elements:

  • 520 RSSI levels
  • Real world coordinates of the sample points (longitude, latitude, floor)
  • Building identifier
  • Reference point (office,…) identifier
  • Relative position with respect to reference point
  • User identifier
  • Phone identifier (model and android version)
  • Timestamp

In this contest, the variables to predict are the coordinates (latitude, longitude and floor identifier) and the building identifier.

The main characteristics of the database are:

  • It covers a surface of 108703m2 including 3 buildings with 4 or 5 floors depending on the building.
  • The number of different places (reference points) appearing in the database is 933.
  • 21049 sampled points have been captured: 19938 for training/learning and 1111 for validation/testing.
  • Dataset independence has been assured by taking Validation (or testing) samples 4 months after Training ones.
  • The number of different wireless access points (WAPs) appearing in the database is 520.
  • Data were collected by more than 20 users using 25 different models of mobile devices.

There is a large private final test set that has not been published. This set will be used to fairly evaluate the Indoor Positionig Systems (IPS) used by the contest participants. New phone models and users can appear on the final test set.

Detailed information regarding the UJIIndoorLoc database can be found in:

J. Torres-Sospedra, R. Montoliu, A. Martínez-Usó, J.P. Avariento, T.J. Arnau, M. Benedito-Bordonau and J. Huerta. UJIIndoorLoc: A New Multi-building and Multi-floor Database for WLAN Fingerprint-based Indoor Localization Problems. Proceedings of the 5th International Conference on Indoor Positioning and Indoor Navigation (IPIN 2014)

 

For any further question about the database, please contact to:

Joaquín Torres (  jtorres(at)uji.es )

Raúl Montoliu (  montoliu(at)uji.es )

Institute of New Imaging Technologies, Universitat Jaume I, Castellón, Spain.

 

How to participate?

Contest registration

In addition to the short (2 to 4 pages) technical description of their system (including the performance of the proposal with the public validation set according to the measure detailed below), participants in the contest should send an email to Joaquín Torres and Raúl Montoliu for contest registration.

Database download

Participants can download the UJIIndoorLoc database at:

https://archive.ics.uci.edu/ml/datasets/UJIIndoorLoc

Final Test set release

Organizers will send to previously registered contest participants the final test set without labels (longitude,latitude,FloorID, and BuildingID).

This test set will be provided after the notification of admission and invitation to submit a paper. The tentative date is 17 June 2015.

Results submission

In addition to the final paper, accepted participants must submit the results on the final test set via email to Joaquín Torres and Raúl Montoliu. Each submission must include a Comma Separated Value (csv) text file with the predicted locations for the final test set. The csv file must have the same number of lines that samples exist in the final test set. Each line should have the following format:

longitude,latitude,FloorID,BuildingID

The results provided in the i-th line should correspond to the i-th sample of the test set (located at i-th line in the test file provided by organizers).

  • Longitude/Latitude format must correspond to the provided in the final test set. Please, maintain the projection & coordinate system, and use the dot symbol “.” as decimal separator. Please, use 16 decimals to represent the longitude and latitude coordinates.
  • FloorID can be 0, 1, 2, 3 or 4.
  • BuildingID can be 0, 1 or 2.

A participant team is able to upload up to 5 different contributions, which will be evaluated by the competition organizers. Although the five alternatives will be evaluated on the final test set, only the best one will be considered for the contest.

The paper that describes the Indoor Positioning Algorithm developed must be written according the IPIN2015 document format, and it must be submitted through EDAS site: https://edas.info/newPaper.php?c=20021

Evaluation

The final metric will be based on:

  • the accuracy on correctly detecting the correct building
  • the accuracy on correctly detecting the correct floor
  • the error in positioning (meters) from the actual and estimated position

In particular, the error for comparing the different IPS will be based on the following equations:

Method accuracy      = average (SampleError(Ri, Ei)),  ∀ samples in the final test set

SampleError(Ri, Ei) = Distance(Ri, Ei) + (penalty1 * buildingfail) + (penalty2 * floorfail)

Where:

  • Ri is the actual position.
  • Ei is the predicted position by the method proposed by the contest participant.
  • buildingfail is 1 if the building prediction is wrong, 0 otherwise.
  • floorfail is the absolute difference between actual floor and the predicted one.
  • penalty1 is used to penalize errors in estimating the building. penalty1 is set to 50.
  • penalty2 is used to penalize errors in estimating the floor. penalty2 is set to 4.
  • Distance(Ri, Ei) calculates the Euclidean distance between coordinates (longitude and latitude) of Ri and Ei.

Conference Session

The best papers will be accepted to participate in the IPIN 2015 conference in a special session. They will present their methods and results on the public test set.

During the session’s closing, we will show the results of the contest and the accuracy of the methods in the private test set.