Forum:Request help/No more relation-data in the backup-files?

From OpenGeofiction
Forums → Request help or feedback → Request help/No more relation-data in the backup-files?


Hello all! Once in a while i download the latest backup (ogf-planet.osm.pbf) from this page https://data.opengeofiction.net/backups/ . With osmconvert i clip it to the area of my country of Neberly and i have some more scripts to get some data out of it. But with the files i downloaded in august and this morning i noticed there is no more data on relations in them. The files i had before (last from november 2021) did had relation-data. My question is then what happened? Is it on purpose that relation-data is no longer included? Is it somewhere else to find? Or is it some hick-up in te last months waiting to be repaired? Does someone know? Antoon (talk) 09:11, 1 October 2022 (UTC)

Are you sure your tools are up-to-date? I use the backup files all the time, and they have relation data in them. I just did a quick test on a fresh setup:
$ wget https://data.opengeofiction.net/backups/ogf-planet.osm.pbf
$ osmconvert ogf-planet.osm.pbf -o=gobras.osm.pbf -B=gobras.poly --complex-ways
# I can load this OSM.PBF file in JOSM, it has 2253 relations in it
$ osmconvert --version | head -2
osmconvert 0.8.10  Parameter Overview
$ osmconvert gobras.osm.pbf -o=gobras.o5m
$ osmfilter gobras.o5m --drop-nodes --drop-ways --out-count=type
       1512     multipolygon
        399     route
        159     boundary
        103     restriction
         28     route_master
         18     land_area
         10     site
          8     building
          1     collection
          1     relation
          1     treaty
The tools used to create the backup changed around April 2022, older utils might not support reading the newer files? wangi (talk) 13:07, 1 October 2022 (UTC)
Thanks for your quick response Wangi. Then probably the change from april has resulted in it for me no longer able to get the relations out of the ogf-planet.osm.pbf file in the way i'm used too for many years. I downloaded the osmconvert.exe file, probably around 2015, and made a step by step plan to first clip the file to the Neberly-area resulting in a pbf-file and that converting to an osm-file. The osm.pbf-file and pbf-file i can't open to check on relations, but when opening the final osm-file i can see no more relations mentioned anymore.
I've tried before to find a new osmconvert-program but failed. The Windows-links on the Openstreetmap-site don't work or lead to a Russian site i don't trust. Wangi, you gave some syntaxes/codes but i have no idea on where or how to use such things. I would be grateful is someone could help me out, but then please keep it plain and simple. Antoon (talk) 12:07, 2 October 2022 (UTC)
I just ran a quite old version of osmconvert (installed around 2018 - the ogfsrtm server is intentionally out-of-date) with a brand-new ogf-planet.osm.pbf, using a gobras.poly - I got the same result as wangi (counts are different because I probably have a more expansive .poly file)
[14:49:14] luciano@ogfsrtm:/opt/geofictician/planet-data/ogieff$ osmconvert ogf-planet.osm.pbf -o=gobras.osm -B=gobras.poly --complex-ways
[14:55:11] luciano@ogfsrtm:/opt/geofictician/planet-data/ogieff$ osmfilter gobras.osm --drop-nodes --drop-ways --out-count=type
       2373	multipolygon
       1302	restriction
        621	route
        515	boundary
         40	route_master
         24	land_area
         10	site
          8	building
          6	turnlanes:turns
          2	network
          2	transit
          2	turnlanes:lengths
          1	bridge
          1	collection
          1	fixme
          1	relation
          1	statistical_area
          1	treaty
To add @Antoon: could you share the steps you take to get your extract? One thought - an incomplete xml (.osm.pbf) might "work" in osmconvert but be missing relations - those are at the end of the file, normally--Luciano (talk) 15:06, 2 October 2022 (UTC)
Hi Luciano. i've written down the steps in a World-file, so here they are...translated from Dutch:

1. Download the latest .osm.pbf-bestand from the following url: https://ogfdata.rent-a-planet.com/backups/ Check on the size of the file as it should be at least the size of former files.

2. Save this file in the following map D:\Antoon\Neberly\Osmconvert and rename it to ogf.osm.pbf (if there's an older file with the same name, rename that one by adding the date to the name)

3. Open the software program osmconvert.exe in the map D:\Antoon\Neberly\Osmconvert, a kind of command prompt will be opened.

4. Type in this command prompt only the letter ‘a’ and press enter. Then there is a text from Bert to lead you through the software program.

5. Type the name of the file to be converted ‘ogf.osm.pbf’ and press enter. Then there is a list of choices of actions.

6. Type the figure 3 and press enter, to point out that (with a border box (coordinates) the file will be reduced to only Neberly.

7. Type the ‘minimum longitude’ 111 and press enter.

8. Type the ‘minimum latitude’ 54 and press enter.

9. Type the ‘maximum longitude’ 135 and press enter.

10. Type the ‘maximum latitude’ 60 and press enter.

Immediately a process will run to create the clipped pbf-file. This will take around 30 seconds and gives the file ogf.osm_01.pbf in the same map. (this file will be around 120x smaller then the original file)

11. Close osmconvert.exe by pressing enter.

12. Rename the file ogf.osm_01.pbf to neberly.pbf. (if there's an older file with the same name, rename that one by adding the date to the name)

13. Open again the software program osmconvert.exe in the map D:\Antoon\Neberly\Osmconvert.

14. Type in this command prompt only the letter ‘a’ and press enter.

15. Type the name of the file to be converted ‘neberly.pbf’ and press enter. Then there's again a list of choices of actions.

16. Type now the figure 1 and press enter, to point out this file needs to be converted to a different type of file.

17. Type again the figure 1 and pressen enter, to point out it needs to be saved as a .osm-file.

Immediately a process will run to create the osm-file. This will take around 1 second and gives the file neberly_01.osm in the same map. (this file will be around 25x larger then the .pbf-file)

18. Close osmconvert.exe by pressing enter.

19. Rename the file neberly_01.osm to neberly.osm. (if there's an older file with the same name, rename that one by adding the date to the name)

In this osmconvert.exe is a small computer program (280kb) i downloaded, probably in 2015. If there's a newer version, please tell me where and how to use it. Antoon (talk) 16:32, 2 October 2022 (UTC)
The link on the official OSM wiki for downloading osmconvert.exe (for Windows) is this: http://m.m.i24.cc/osmconvert32 - I followed this link, and I got a warning about "unsafe source" but ignoring that (it's published on the official OSM wiki!) the program downloaded fine. I put it on an old Windows machine I have. It runs and behaves as I would expect osmconvert to behave. However, it complains about "memory error" - but I think that's because my Windows machine is old and weak. Anyway I think you should go ahead trust that download source and get an updated version. I hope that solves the problem. (Today I learned: osmconvert has a user interface. I never knew.)--Luciano (talk) 19:36, 2 October 2022 (UTC)
Thanks for your answer Luciano, it helped me get an osm-file from Neberly, now with relation-data. For me it had to be http://m.m.i24.cc/osmconvert64 as i'm running Windows in 64bit but it downloaded the osmconvert-program in which i went through the same steps as mentioned yesterday. Both the PBF- and the OSM-files are a bit larger then with the old osmconvert-program. And when opening the OSM-file i find the relation-tags in it. Tomorrow i will run further scripts to see if i can really use it, but it looks as it has worked for now...so again: thanks! Antoon (talk) 15:05, 3 October 2022 (UTC)
Retrieved from "https://wiki.opengeofiction.net/index.php?title=Forum:Request_help/No_more_relation-data_in_the_backup-files%3F&oldid=12638"