Osm2pgsql/benchmarks/Dell R610 import 2

From OpenStreetMap Wiki
Jump to: navigation, search

osm2pgsql command line :

$ ../osmconvert/osmconvert planet.osm.pbf --out-osm |  ../osm2pgsql/osm2pgsql --flat-nodes /data/work/osm2pgsql/flat-nodes.raw --number-processes=4 -k -C 19000 -s -S ./default.style -G -m -d osm2pgsql /dev/stdin
osm2pgsql SVN version 0.80.0 (64bit id space)

Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE:  table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE:  table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE:  table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE:  table "planet_osm_roads_tmp" does not exist, skipping
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=19000MB, maxblocks=2432001*8192, allocation method=11
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Allocated space for persistent node cache file
Maximum node in persistent node cache: 0
Mid: pgsql, scale=100 cache=19000
Setting up table: planet_osm_nodes
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_nodes_pkey" for table "planet_osm_nodes"
Setting up table: planet_osm_ways
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_ways_pkey" for table "planet_osm_ways"
Setting up table: planet_osm_rels
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_rels_pkey" for table "planet_osm_rels"

Reading in file: /dev/stdin
Processing: Node(1722186k 205.1k/s) Way(164448k 17.57k/s) Relation(1738047 131.08/s)  parse time: 31015s

Node stats: total(1722186875), max(2091687036) in 8396s
Way stats: total(164448446), max(199128620) in 9360s
Relation stats: total(1738047), max(2674511) in 13259s
Committing transaction for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_polygon
Committing transaction for planet_osm_roads

Going over pending ways...
Maximum node in persistent node cache: 2091909119
        97244643 ways are pending
Using 4 helper-processes
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
processing way (0k) at 0.00k/s (done 0 of 4)Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
processing way (0k) at 0.00k/s (done 0 of 4)Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
Process 0 finished processing 24311161 ways in 7949 sec
Process 2 finished processing 24311161 ways in 7949 sec
Process 1 finished processing 24311161 ways in 7949 sec
Process 3 finished processing 24311160 ways in 7949 sec
Maximum node in persistent node cache: 2091909119
Maximum node in persistent node cache: 2091909119
Maximum node in persistent node cache: 2091909119

All child processes exited

97244643 Pending ways took 7977s at a rate of 12190.63/s
Committing transaction for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_polygon
Committing transaction for planet_osm_roads

Going over pending relations...
Maximum node in persistent node cache: 2091909119
        0 relations are pending

Using 4 helper-processes
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
Maximum node in persistent node cache: 2091909119
Process 1 finished processing 0 relations in 14 sec
Process 2 finished processing 0 relations in 14 sec
Maximum node in persistent node cache: 2091909119
Maximum node in persistent node cache: 2091909119
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
Mid: loading persistent node cache from /data/work/osm2pgsql/flat-nodes.raw
Maximum node in persistent node cache: 2091909119
Process 0 finished processing 0 relations in 14 sec
Process 3 finished processing 0 relations in 14 sec
Maximum node in persistent node cache: 2091909119

All child processes exited
0 Pending relations took 15s at a rate of 0.00/s

node cache: stored: 1722186875(100.00%), storage efficiency: 86.60% (dense blocks: 1823124, sparse nodes: 60855638), hit rate: 100.00%
Sorting data and creating indexes for planet_osm_roads
Sorting data and creating indexes for planet_osm_line
Sorting data and creating indexes for planet_osm_polygon
Sorting data and creating indexes for planet_osm_point
Maximum node in persistent node cache: 2091909119
Stopping table: planet_osm_ways
Stopping table: planet_osm_nodes
Building index on table: planet_osm_ways (fastupdate=off)
Stopping table: planet_osm_rels
Building index on table: planet_osm_rels (fastupdate=off)
Stopped table: planet_osm_nodes in 0s
Stopped table: planet_osm_rels in 197s
Analyzing planet_osm_roads finished
Analyzing planet_osm_point finished
Analyzing planet_osm_polygon finished
Analyzing planet_osm_line finished
Copying planet_osm_roads to cluster by geometry finished
Creating indexes on  planet_osm_roads finished
All indexes on  planet_osm_roads created  in 2634s
Completed planet_osm_roads
Copying planet_osm_point to cluster by geometry finished
Creating indexes on  planet_osm_point finished
All indexes on  planet_osm_point created  in 6689s
Completed planet_osm_point
Copying planet_osm_polygon to cluster by geometry finished
Copying planet_osm_line to cluster by geometry finished
Creating indexes on  planet_osm_polygon finished
All indexes on  planet_osm_polygon created  in 17421s
Completed planet_osm_polygon
Creating indexes on  planet_osm_line finished
All indexes on  planet_osm_line created  in 17496s
Completed planet_osm_line
Stopped table: planet_osm_ways in 55370s

Osm2pgsql took 94448s overall