Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gisDeliveryWF load-vector: refine choice of encoding for shp2pgsql call? #850

Open
jmartin-sul opened this issue Mar 5, 2024 · 1 comment

Comments

@jmartin-sul
Copy link
Member

jmartin-sul commented Mar 5, 2024

A comment from Robots::DorRepo::GisDelivery::LoadVector#perform_work used to say (I replaced that comment with a link to this issue):

            # encoding =  # XXX: these are hardcoded encodings for certain druids -- these should be read from the metadata somewhere
            #   case druid
            #   when 'bt348dh6363', 'cc936tf6277'
            #     'LATIN1'
            #   else
            #     'UTF-8'
            #   end

Instead of figuring out the encoding based on metadata for the object per the above suggestion, the following fall-through approach is used (as of march 2024), assuming that an error in the shp2pgsql call is a result of using the wrong encoding:

# first try decoding with UTF-8 and if that fails use LATIN1
# see also https://github.com/sul-dlss/gis-robot-suite/issues/850
begin
run_shp2pgsql('4326', 'UTF-8', shp_filename, schema, sql_filename, stderr_filename)
rescue RuntimeError => e
logger.warn("#{druid} -- fell through to LATIN1 encoding after calling run_shp2pgsql with " \
"UTF-8 encoding and encountering error: #{e.message}; #{e.backtrace.join('; ')}")
run_shp2pgsql('4326', 'LATIN1', shp_filename, schema, sql_filename, stderr_filename)
end

Is this something that we should refine, per the old comment's suggestion? Do we ever fall through to use the LATIN1 encoding anymore? This question was introduced in a commit from late 2014, so it may well be irrelevant now. See 6eff636

bumped into this working on #851

@jmartin-sul
Copy link
Member Author

This exception handling for choosing the right encoding is still useful functionality, though it changed again very slightly, see #883

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant