Geotagged meta data

I (User:Brian Schimmel) am currently planing some major changes to JOSM and the way it handles meta data. Before jumping into the source code, I want to make some general assumptions on geotagged meta data. I want to share them with you, so you can get a deeper insight in how JOSM will propably work in the near future, and for you to find errors and incompetions in my thoughts. If you are looking for some how-to's, then you are at the wrong place.

What is geotagged meta data?

What is meta data?

There are many general definitions, but for the scope of OSM, I would like to define meta data as data that will not be included as is in the main database (this would just be called data), but that you may want to show in your map editor (e.g. JOSM). This may be

• photos, paintings
• text notes
• voice
• video
• maybe others

What is geotagged?

Geotagged means, that the information concerns a certain location on the real world, and there is digital information about where this place is. Most of the times, the location will not be known directly, but there is information about the time of creation of the meta data. Together with some kind of time-to-space-mapping the location of the meta data can be inferred. This is the simple version, treating the meta data as if it belonged to a single point in the world.

The data may continous, that is, have a duration in time. This applies mainly to audio and video. Provided that the mapper was moving while he recorded, there is some subset of his path that belongs to the media. Not only this subset is of interest, but for each frame of the audio/video, you might want to know the corresponding location.

In addition to the location, there may be directional information. Think of geotagged photos and a camera with an internal compass. These devices are very rare (though I heard they already exist), but I would like to be prepared for this kind of information.

What about satellite images and other information from WMS servers?

Per definition, this is also geotagged meta data. But it differs enough from what I have in mind now, to let it out of scope.

But there may be something in between: Imagine you are standing on some high point (maybe a tower or cliff) and take a (geotagged, of course!) photograph of what is below you. If there were three visible features in the photo, which are already in the map, then you would be able to rectify this photo and use it as a overlay as you do with satellite images. Current consumer cameras have up to 12 megapixels, enoug to catch each detail. Despite all the possible problems that might arise: This is geotagged meta data as defined above. The line to satellite images is very thin here.

Time

There is only ONE time

If you ignore all relativistic effects (and i guess you are not mapping near light speed!) we could agreee that there is a single time or clock. The UTC is widly accepted as this. Each GPS device gets the GPS-time, along with all needed information to compute the UTC with less than a millisecond error. So your GPS tracks will always have the proper UTC timestamps. Pherhaps there are some GPS that do not convert GPS-time to UTC. The difference is currently (2008) 14 seconds, so you probably will not have noticed if your device does this.

But there may be other times... :)

The meta data you record will have some timestamp attached. If it comes from the same clock (that is, from you GPS receiver), time is defined equally. If it is syncronized to you local time, lets say via internet (using a proper time protocol) or via radio, then it will have the same seconds and same minutes, but different hours due to time zones. There are also timezones that are off by 30 minutes to their neighbours. Most countries have something called Daylight saving time which adds another hour of difference to this. JOSM currently handles these issues in some way, but it is very confusing for the user.

Most of your devices will have their own clock. Ideally, they would be exactly synchronous. But in reality, there may be differences:

• Clock runs to slow or to fast
• The device does not know about Daylight saving time. If you adjust it manually, but do not do the opposite in autumn, you get an hour difference in the other direction.
• The device does not know about Febuary 29th, and thus has the right time but wrong date
• The device's clock was set when you were in a different time zone
• The device syncs with some other device/server/technique which has any of the above errors

How to handle this: Save time for every device

I would like to store all times internally in UTC. There should not be any persistent Date-object based on some other time. To make the user happy, I would then like to present each and every timestamp in his local time. Java has some internal mechanisms to make this happen. When times are written into files, or read from a file, and this file travels from one user to the other, we should use UTC and make clear that it is used. If we have same reason to use another time base, just make it as clear as possible.

The user will not want to use UTC on all his devices, but he will want to use local time. Maybe we can help him to get the proper local time including seconds onto his device.

But anyway, users will continue creating geotagged meta data with inaccurate or wrong times. So lets keep some list of his devices, along with the information wich media types they record. For example my list would look like this:

• V600i (mobile phone): audio, video, photo, text, time
• Siemens (mobile phone): audio, text, time
• Dell M70 (Laptop): audio, text, time
• OQO model 01+ (Ultra portable Tablet PC): audio, text, time, drawing
• Green Wristwatch (clock): time
• Silver Wristwatch (clock): time
• Megapix VX8 (Digital camera): audio, video, photo
• Zeitnahmegerät HS-Harz unten (clock): time
• Zeitnahmegerät HS-Harz oben (clock): time
• PC Pool HS-Harz (clock): time

For each device, we save the offset to the UTC. When the user imports metadata, he has to select from wich device the metadata comes. To make his life easyer, the list will be prefiltered depending on the type of media. To be more precise, he has to chose from where his time comes. Why that? My two mobile phones do not show a clock while they record audio, and they do not save proper timestamps. So I have to say aloud the time to have kind of a timestamp. I read the time from some other device, thus creating audio with device A, tagging it with a time from device B. This might explain why I include my wristwatches and other clocks, even if they cannot create metadata by themselves.

Note: it might to be interesting what happens if I use some JOSM GPS plugin to use the live position: Does it take the GPS time (this is: UTC) or the time of my laptop?

Sadly, many deviced exist that have very exact clocks, but do not show the seconds to the user. I use these sometimes (if nothing better is around). I might include this devices to the list, marking them as minute-clock, to reming me and the computer that the geotagged data has to synced in some way. The important thing here is: We have to sync every piece of media on its own. It will not be enough to define "This device is -21 seconds off." as this will change from piece to piece.

Space

Usually, the mapper will record his location with a GPS device in some form of track log, preferably in GPX format. This will be a list of locations, each one timestamped. Currently, JOSM creates one GPX layer per GPX file, and when you import photos, you have to decide to wich GPX file they belong to. Why doesn't JOSM do this automatically? As long as the time spans of the GPX files do not overlap, this should not be a problem.

So I suggest there should be some thing (a class, object, mehtod...) in JOSM that can be given a time and answers with a location. It should be some abstraction layer over GPX files and other location/time providers.

Untagged meta data

There must be some easy way to handle meta data that has no geodata that can be interpreted by the software. And I should think about how GPS Waypoints fit into this sheme.

User interface

I think the user should be able to import any kind of data the same way, even different types at the same time. And he should be free wether to import the meta data or the positions first. Maybe JOSM keeps track of what files were already processed, so he can just configure in which folder the data is to be found, and click a button "load new meta data media".

JOSM should then show all the media as icons (or some other way) in the map. If it is needed to convert the data (as AMR audio recordings from mobile phones), it should check if it was done before, and do it in the background, showing results as the get ready. Also, it should ask the user for his timesources. They may be different for the many items. He will need some preview method to check what image/audio/text it is.

As soon as all media is importetd, time- and geotagged, the user will want to map. He may navigate though all his media by time. There will be some playhead/cursor advancing in time, with a real-time speed. This may be shown on a slider/progress bar, as well as a traveling spot in the map. Audio or video is played synchronous. The photo/drawing/text whose timestamp is nearest to the playtime will be shown. Maybe the user needs some timeline where he can see when he took a phote, a note, video, audio... He could use this to easily jump in time.

There should be global shortcuts for play, pause, jump back 10 seconds, jump to the next/previous file. These should be active even when he is inside a always-on-top dialog. Is it possible to react on multimedia shortcut keys as they are found on most USB keyboards?

Session control

I'm not sure if JOSM already has this, but it would be nice to have some way to save the current state (which files are loaded, which parts of the map are shown) to continue work where you leave it.