locnguyen (3) [Avatar] Offline
#1
I run this command in Terminal but get a malloc error from shp2pgsql:

hackint0shsmilieesktop locnguyen$ shp2pgsql -G -g geog /Users/locnguyen/Desktop/data/roadtrl020.shp ch01.roads_geog | psql -h localhost -U locnguyen -p 5432 -d testgis

Result:

shp2pgsql(656) malloc: *** error for object 0x7fff5fbff570: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

I'm a total n00b to PostGIS and GIS in general so I'm a little lost as to what the problem might be.
rhurlin (9) [Avatar] Offline
#2
Re: [CH01] 1.5 - Error loading into geog type on cmd line
I can confirm this behaviour for the command line version of shp2pgsql on FreeBSD operating system, version 9.0-CURRENT amd64 (www.freebsd.org).

When trying to convert a shapefile with shp2pgsql from database/postgis version 1.5.2 it core dumps when using the two options '-G' and '-g' at the same time. This error is reproducable with other shapefiles, too.

----------------------
shp2pgsql -G -g geog_newname world.shp
shp2pgsql: (malloc) /usr/src/lib/libc/stdlib/malloc.c:4287: Failed assertion: "arena != NULL"
Abbruch(core dumped)
----------------------

It seems there is a problem with memory allocation. This does not happen, if '-G' is used without the '-g' option and it does not happen when using the gui version shp2pgsql-gui.

Rainer
locnguyen (3) [Avatar] Offline
#3
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Thank you for the help. I hope this bug has been reported?
rhurlin (9) [Avatar] Offline
#4
Re: [CH01] 1.5 - Error loading into geog type on cmd line
No, I did not. But you are right, we should do.

As far as I can see in the loader sources there is a maintainer address. I will give it a try in the next hours.

Rainer
regina.leo (265) [Avatar] Offline
#5
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Which version of 1.5 shp2pgsql are you running? I recall this being a fixed issue with upcoming 2.0, but don't recall it being an issue with 1.5 shp2pgsql

http://trac.osgeo.org/postgis/ticket/659

To verify which shp2pgsql you are running, just call it from command line with no args or with --h

like

shp2pgsql

Thanks,
Regina
rhurlin (9) [Avatar] Offline
#6
Re: [CH01] 1.5 - Error loading into geog type on cmd line
On FreeBSD as I wrote before, PostGIS version 1.5.2

shp2pgsql: illegal option -- h
RCSID: $Id: shp2pgsql-core.h 5983 2010-09-19 11:27:05Z mcayland $ RELEASE: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1

Rainer
regina.leo (265) [Avatar] Offline
#7
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Hmm okay. I suspect its the same issue as we had in 2.0. I hadn't thought the issue existed in 1.5, but perhaps it just didn't exhibit itself under windows. I think some other fixes were made in 1.5.3 so you might want to try that. That was released about 3 weeks ago. In the mean time I add the link to this thread on the original ticket I had posted for 2.0 that I suspect is the same issue.
rhurlin (9) [Avatar] Offline
#8
Re: [CH01] 1.5 - Error loading into geog type on cmd line
I just tried PostGIS 1.5.3, see

# SELECT postgis_full_version();
POSTGIS="1.5.3" GEOS="3.3.0-CAPI-1.7.0" PROJ="Rel. 4.7.1, 23 September 2009" LIBXML="2.7.8" USE_STATS (procs from 1.5 r5976 need upgrade)

It seems there is some upgrade work to do inside the database ...


The loader gives me exact the same error as in 1.5.2. As far as I can see its even exact the same version as in 1.5.2 smilie

# shp2pgsql -h
RCSID: $Id: shp2pgsql-core.h 5983 2010-09-19 11:27:05Z mcayland $ RELEASE: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1

Rainer
regina.leo (265) [Avatar] Offline
#9
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Okay I'll ticket this bug. The stamp only records when shp2pgsql-core.h is changed, but sadly other parts aren't captured so the timestamp is not absolutely definitive for micro releases. More a rule of thumb.
rhurlin (9) [Avatar] Offline
#10
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Thanks for the ticket. I have to mention that I also wrote a mail to the maintainer Paul Ramsey today. No answer so far.

I made a quick diff between loader section of 1.5.2 and 1.5.3 sources. Indeed there are some 'micro' changes:

-------------------------------------------------------------------
diff -Naur postgis-1.5.2/loader postgis-1.5.3/loader
diff -Naur postgis-1.5.2/loader/Makefile postgis-1.5.3/loader/Makefile
--- postgis-1.5.2/loader/Makefile 2010-09-27 17:19:08.000000000 +0200
+++ postgis-1.5.3/loader/Makefile 2011-06-25 18:29:50.000000000 +0200
@@ -16,8 +16,8 @@
# the INSTALL and DESTDIR variables so we can get the correct install paths.
# Hence we include the PGXS Makefile here, but ensure that we override the
# 'all' and 'install' targets with the ones we really want to use below.
-PG_CONFIG = /usr/local/pgsql/8.3/bin/pg_config
-PGXS := /usr/local/pgsql/8.3/lib/pgxs/src/makefiles/pgxs.mk
+PG_CONFIG = /usr/local/pgsql/9.1/bin/pg_config
+PGXS := /usr/local/pgsql/9.1/lib/pgxs/src/makefiles/pgxs.mk
include $(PGXS)

# Set CFLAGS afer PGXS, otherwise it will get overwritten with the PGXS
@@ -31,8 +31,8 @@
SHP2PGSQL-GUI=shp2pgsql-gui

# PostgreSQL frontend CPPFLAGS and LDFLAGS (for compiling and linking with libpq)
-PGSQL_FE_CPPFLAGS=-I/usr/local/pgsql/8.3/include
-PGSQL_FE_LDFLAGS=-L/usr/local/pgsql/8.3/lib -lpq
+PGSQL_FE_CPPFLAGS=-I/usr/local/pgsql/9.1/include
+PGSQL_FE_LDFLAGS=-L/usr/local/pgsql/9.1/lib -lpq

# iconv flags
ICONV_LDFLAGS= -liconv
diff -Naur postgis-1.5.2/loader/shp2pgsql-cli.c postgis-1.5.3/loader/shp2pgsql-cli.c
--- postgis-1.5.2/loader/shp2pgsql-cli.c 2010-03-22 20:38:40.000000000 +0100
+++ postgis-1.5.3/loader/shp2pgsql-cli.c 2010-12-14 05:57:59.000000000 +0100
@@ -1,5 +1,5 @@
/**********************************************************************
- * $Id: shp2pgsql-cli.c 5451 2010-03-22 19:38:40Z pramsey $
+ * $Id: shp2pgsql-cli.c 6373 2010-12-14 04:57:59Z pramsey $
*
* PostGIS - Spatial Types for PostgreSQL
* http://postgis.refractions.net
@@ -31,17 +31,18 @@
printf(" -c Creates a new table and populates it, this is the
");
printf(" default if you do not specify any options.
");
printf(" -p Prepare mode, only creates the table.
");
- printf(" -g <geocolumn> Specify the name of the geometry/geography column
");
- printf(" (mostly useful in append mode).
");
- printf(" -D Use postgresql dump format (defaults to SQL insert statments.
");
+ printf(" -g <geocolumn> Specify the name of the geometry/geography column.
");
+ printf(" (mostly useful in append mode).
");
+ printf(" -D Use postgresql dump format (defaults to SQL insert statments).
");
printf(" -G Use geography type (requires lon/lat data).
");
printf(" -k Keep postgresql identifiers case.
");
printf(" -i Use int4 type for all integer dbf fields.
");
printf(" -I Create a spatial index on the geocolumn.
");
printf(" -S Generate simple geometries instead of MULTI geometries.
");
+ printf(" -w Output WKT format (drops M and introduces coordinate drifts).
");
printf(" -W <encoding> Specify the character encoding of Shape's
");
- printf(" attribute column. (default : "WINDOWS-1252")
");
- printf(" -N <policy> NULL geometries handling policy (insert*,skip,abort)
");
+ printf(" attribute column. (default : "WINDOWS-1252").
");
+ printf(" -N <policy> NULL geometries handling policy (insert*,skip,abort).
");
printf(" -n Only import DBF file.
");
printf(" -? Display this help screen.
");
}
diff -Naur postgis-1.5.2/loader/shp2pgsql-core.c postgis-1.5.3/loader/shp2pgsql-core.c
--- postgis-1.5.2/loader/shp2pgsql-core.c 2010-09-19 18:54:59.000000000 +0200
+++ postgis-1.5.3/loader/shp2pgsql-core.c 2010-12-13 21:42:47.000000000 +0100
@@ -1,5 +1,5 @@
/**********************************************************************
- * $Id: shp2pgsql-core.c 5989 2010-09-19 16:54:59Z mcayland $
+ * $Id: shp2pgsql-core.c 6361 2010-12-13 20:42:47Z pramsey $
*
* PostGIS - Spatial Types for PostgreSQL
* http://postgis.refractions.net
@@ -1587,6 +1587,7 @@
char *geometry=NULL, *ret;
char *utf8str;
int res, i;
+ int rv;

/* Clear the stringbuffers */
sbwarn = stringbuffer_create();
@@ -1655,7 +1656,8 @@
{
case FTInteger:
case FTDouble:
- if (-1 == snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i)))
+ rv = snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i));
+ if (rv >= MAXVALUELEN || rv == -1)
{
vasbappend(sbwarn, "Warning: field %d name truncated
", i);
val[MAXVALUELEN - 1] = '';
@@ -1676,7 +1678,8 @@
case FTString:
case FTLogical:
case FTDate:
- if (-1 == snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i)))
+ rv = snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i));
+ if (rv >= MAXVALUELEN || rv == -1)
{
vasbappend(sbwarn, "Warning: field %d name truncated
", i);
val[MAXVALUELEN - 1] = '';
regina.leo (265) [Avatar] Offline
#11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
I wouldn't call Paul THE maintainer. It's maintained by all of the PostGIS development crew so the proper protocol for these type of things is to submit a bug ticket via PostGIS bug tracker.

http://trac.osgeo.org/postgis/

Or a question on Postgis news group if you are not sure it's a bug.

http://www.postgis.org/support/

Paul responds more quickly to those than his own email smilie
rhurlin (9) [Avatar] Offline
#12
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Ahh, I guessed so. OK, next time I will first ask on the mailing list.

In ticket #1132 you only refer to my report for FreeBSD. The original poster Loc Nguyen seems to be on Mac OSX or something like that ...
locnguyen (3) [Avatar] Offline
#13
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Indeed I'm running Mac OS X. Running shp2pgsql without arguments:

hackint0sh:~ locnguyen$ shp2pgsql
RCSID: $Id: shp2pgsql-core.h 5983 2010-09-19 11:27:05Z mcayland $ RELEASE: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1


If it helps I'm running 1.5.2

POSTGIS="1.5.2" GEOS="3.2.2-CAPI-1.6.2" PROJ="Rel. 4.7.1, 23 September 2009" LIBXML="2.7.3" USE_STATS
regina.leo (265) [Avatar] Offline
#14
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Okay I've updated. I'll have to try again on windows but not near my computer with shp2pgsql