Efficiently indexing with offset bitmaps for huge sets of slightly disordered sensor data

Takahiro Yamazaki, Takeru Inoue, Hiroshi Sato, Noriyuki Takahashi, Junichiro Takagi, Masateru Minami

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

With advances in sensor devices and networking technologies, it is expected that future networks will contain immense numbers of sensors that are capturing time-varying data. It is necessary to process queries over the data for analysis and to store the data for later use. For querying the data, data disorder is a common problem. Existing approaches use buffers to recover the order but there are problems in deciding buffer sizes and buffer residence times. To avoid these problems and to satisfy the need to store the data for later use, our approach writes indexed data rapidly to a stable disk. This approach, however, has a problem in that existing indexing methods are not up to the task; "sparse indexing" offers fast write-speeds but cannot handle disordered data, "dense indexing" can handle disordered data but its write-speed is inadequate. Our solution is a novel indexing method that can rapidly store slightly disordered time-series sensor data. Our proposal succeeds because it uses a novel data structure, "offset bitmaps", to extend the sparse indexing method. The offset bitmaps provide pointers that can handle delayed data and so allow them to be managed efficiently. Experiments show that our method has several advantages over conventional alternatives. We verify that the proposed method has 10 times the write speed of the existing dense indexing method.

Original languageEnglish
Title of host publication8th Asia-Pacific Symposium on Information and Telecommunication Technologies, APSITT 2010
Publication statusPublished - 2010 Sep 16
Event8th Asia-Pacific Symposium on Information and Telecommunication Technologies, APSITT 2010 - Kuching, Malaysia
Duration: 2010 Jun 152010 Jun 18

Publication series

Name8th Asia-Pacific Symposium on Information and Telecommunication Technologies, APSITT 2010

Conference

Conference8th Asia-Pacific Symposium on Information and Telecommunication Technologies, APSITT 2010
CountryMalaysia
CityKuching
Period10/6/1510/6/18

Keywords

  • Component
  • Data structure
  • Indexing
  • Sensor network

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Efficiently indexing with offset bitmaps for huge sets of slightly disordered sensor data'. Together they form a unique fingerprint.

  • Cite this

    Yamazaki, T., Inoue, T., Sato, H., Takahashi, N., Takagi, J., & Minami, M. (2010). Efficiently indexing with offset bitmaps for huge sets of slightly disordered sensor data. In 8th Asia-Pacific Symposium on Information and Telecommunication Technologies, APSITT 2010 [5532030] (8th Asia-Pacific Symposium on Information and Telecommunication Technologies, APSITT 2010).