event terminology

[dustymc]

But I think this has always been a problem what exactly does specimen/event remark mean? It has never been clear.

Anyway, change this to "Collection Object Remark" to match data entry

Change this to "Verbatim Event Details" to match catalog record (but see below)

Change this to "Verbatim Event Details" to match data entry

Change this to "Catalog Record Event Remark" on the catalog record

Change this to "Catalog Record Event Remark" in data entry

And also make appropriate changes to search. I can do all of this and let everyone see it in test.

Originally posted by @Jegelewicz in #3691 (comment)

[dustymc]

I greatly dislike "Verbatim Event Details" - that "node" is much more than that. The rest I could live with, I think, but it's extremely verbose.

We should also be looking at APIs - they will not share this terminology. (And the above doesn't cover all non-API usage).

Please do not make arbitrary changes; this needs planned.

what exactly does specimen/event remark mean

It is remarks regarding the intersection of specimens - "catalog records" - and the event stack.

arctosprod@arctos>> \d specimen_event
                                              Table "core.specimen_event"
        Column         |            Type             | Collation | Nullable |                  Default                  
-----------------------+-----------------------------+-----------+----------+-------------------------------------------
 specimen_event_id     | integer                     |           | not null | nextval('sq_specimen_event_id'::regclass)
 collection_object_id  | integer                     |           | not null | 
 collecting_event_id   | integer                     |           | not null | 
 assigned_by_agent_id  | integer                     |           | not null | 
 assigned_date         | timestamp without time zone |           | not null | 
 specimen_event_remark | character varying(4000)     |           |          | 
 specimen_event_type   | character varying(60)       |           | not null | 
 collecting_method     | character varying(4000)     |           |          | 
 collecting_source     | character varying(60)       |           |          | 
 verificationstatus    | character varying(60)       |           | not null | 
 habitat               | character varying(4000)     |           |          | 
 verified_by_agent_id  | integer                     |           |          | 
 verified_date         | character varying(30)       |           |          | 

[Jegelewicz]

OK - I have had a few nights of sleep to dream about this and here are some thoughts.

First - I think that if the data entry screen were structured in the same way the catalog record is, there would be less confusion. (and I know it sort-of is, but in a different order)

Beyond that, let's talk about the "event stack"

I'm going to start with the top of the stack - the part where we assign an event to a catalog record, currently known as the "specimen/event" (I have been trying to rid us of specimen)

Here is a comparison of all the ways we refer to sections of this part of the event stack:

Arctos Page/What it does	Assigned Event Type	Event Assigned By	Event Assigned Date	Event Verification Status	Event Verified By	Event Verification Date	Event Assignment Remark	Event Habitat	Event Source	Event Method
Edit catalog record	Specimen/Event Type	Event Determiner	Determined Date	Verification Status	Verified By	Verified Date	Specimen/Event Remark	Habitat	Collecting Source	Collecting Method
Catalog record	Event Type	Event Type: assigned by	Event Date	Event Verification:	Event Verification: verified by	Event Verification: on	Event Details: remark	Event Details: habitat	Event Details: Collecting Source	Event Details: Collecting Method
Data entry (old)	Specimen/Event Type	Event Determiner	Detr. Date	VerificationStatus	N/A	N/A	Specimen/Event Remark	Habitat	Coll. Src.	Coll. Meth.
Data entry (new)	Event Type	Event Determiner	Det. Date	VerificationStatus	N/A	N/A	Event Remark	Habitat	Collecting Source	Collecting Method
Bulkloader builder	specimen_event_type	event_assigned_by_agent	event_assigned_date	verificationstatus	N/A	N/A	specimen_event_remark	habitat	collecting_source	collecting_method
Bulkload specimen-events	specimen_event_type	assigned_by_agent	assigned_date	verificationstatus	N/A	N/A	specimen_event_remark	habitat	collecting_source	collecting_method

So I can see why there is confusion but before we start changing stuff, I think we need to take a hard look at what this information is meant to convey and make sure that the labels fit. I'd like to see this whole part of the stack better reflect what it is actually doing - recording information about who, when, and why the associated event/locality are assigned to the object(s) or observation(s) this catalog record describes. If that is a poor description of what is going on, please give me a better one!

FWIW documentation says this (and needs to get rid of "specimen")

Specimen Event
Specimen Events link Specimens to Collecting Events.

A few other musings:

1. It is unclear to me as to why "habitat" falls within this section of the stack and is not a "collecting event" attribute.
2. I am also feeling ambivalent about the placement of "collecting method" and "collecting source", but perhaps that will become more clear when I take a look at the "collecting event" part of the stack....

[dustymc]

data entry screen were structured in the same way the catalog record is

It won't be - one has a fixed structure (with some "extras"), the other is purely data driven and has no limitations.

different order

The entry screen has no order at all (and maybe some day other screens will be similar).

what this information is meant to convey

https://arctos.database.museum/tblbrowse.cfm?tbl=specimen_event

make sure that the labels fit.

That's one of the original use cases for field-level documentation, but we don't follow it (nor do we have the tools to make that mostly-realistic). Maybe that should be merged with the table browser in some way. Also, context - which varies from UI to UI - matters, unless we want to be very verbose everywhere (which brings on a whole new batch of usability problems).

In any case, I'd like to keep more of the very core documentation - "this table does THAT, this field means THING" - in the database itself, where it's available to APIs (including local UI). That's probably yet another Issue....

why "habitat" falls within this section of the stack and is not a "collecting event" attribute.

That's reflective of a lack of precision in recording events; events end up as "today, within a mile of here" and habitat within that event is "under a rotting log" and "bare vertical rock" and .... I doubt we can do much for legacy data, but we can certainly make recommendations to GPS-equipped collectors going forward.

"collecting method" and "collecting source",

Same as above, but also things like host being collected in a pitfall and parasites being collected in a Number Whatever Sieve (so not "fixable" via more precise place-time data).

[Jegelewicz]

data entry screen were structured in the same way the catalog record is

It won't be - one has a fixed structure (with some "extras"), the other is purely data driven and has no limitations.

I just mean the same order - right now locality is where collectors and identifiers are in the catalog record. If we could keep to the same general order, this would help, I think - but this is just one person's opinion.

[dustymc]

locality is where

... ever a user put it.

[Jegelewicz]

Fair enough.

OK, here is the comparison for "collecting event" which is a misnomer, because it can also represent any of the other event types in https://arctos.database.museum/info/ctDocumentation.cfm?table=ctspecimen_event_type

Arctos Page/What it does	Verbatim Event Location	Verbatim Event Date	Formatted Event Start Date	Formatted Event End Date	Event Remark	Event Name	Verbatim Coordinates	Event Attribute Type	Event Attribute Value	Event Attribute Units	Event Attribute Determiner	Event Attribute Date	Event Attribute Method	Event Attribute Remark	Locality Name	Event ID	Locality ID
Edit catalog record	Verbatim Locality	Verbatim Date	Began Date/Time	Ended Date/Time	Collecting Event Remark	Collecting Event Nickname	As-entered Coordinates	Event Attribute Type	Event Attribute Value	Event Attribute Units	Event Attribute Determiner	Event Attribute Date	Event Attribute Method	Event Attribute Remark	locality_name	N/A
Catalog record	Verbatim Locality	Verbatim Date	Event Date	Event Date (to)	Remark	Event Nickname	As Entered	Attribute	Value	included in value if applicable	Determiner	Date	Method	Remark	Locality Nickname	N/A
Data entry (old)	Verbatim Locality	VerbatimDate	Begin	End	CollEvntRemk	Event Nickname	Coordinates (event and locality)	N/A	N/A	N/A	N/A	N/A	N/A	N/A	Locality Nickname	Existing EventID
Data entry (new)	Verbatim Locality	Verbatim Date	Began Date	Ended Date	Collecting Event Remark	Event Name	all the coordinate stuff that is not decimal degrees	N/A	N/A	N/A	N/A	N/A	N/A	N/A	Locality Name	Event ID	Locality ID
Bulkloader builder	verbatim_locality	verbatim_date	began_date	ended_date	coll_event_remarks	collecting_event_name	all the coordinate stuff that is not decimal degrees	N/A	N/A	N/A	N/A	N/A	N/A	N/A	locality_name	collecting_event_id	locality_id
Bulkload specimen-events	verbatim_locality	verbatim_date	began_date	ended_date	coll_event_remarks	collecting_event_name	all the coordinate stuff that is not decimal degrees	N/A	N/A	N/A	N/A	N/A	N/A	N/A	locality_name	N/A	N/A
Bulkload collecting events	verbatim_locality	verbatim_date	began_date	ended_date	coll_event_remarks	collecting_event_name	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	locality_name	N/A	N/A
Bulkload collecting event attributes	N/A	N/A	N/A	N/A	N/A	event_name	N/A	event_attribute_type	event_attribute_value	event_attribute_units	event_determiner	event_determined_date	event_determination_method	event_attribute_remark	N/A	N/A	N/A

A few thoughts on this:

1. Why are we including Event and Locality ID's as pick values when these could so easily change?
2. Where in the heck does "associated species" fall in all of this? Is it part of the event stack or not?
3. Why can't I enter event attributes as "extras"? It seems like we are discouraging expanding information related to the event because the only way to enter it is manually or through an event attribute bulkload (which assumes you have named events, which you cannot create in data entry). If this is technically too difficult, then OK, but if I can enter locality attributes as extras, it seems like event attributes should be no different.

[dustymc]

misnomer

No argument, but that's Major Surgery (and I'm not sure I have better names).

Why are we including Event and Locality ID's as pick values when these could so easily change?

@campmlc had some compelling reason, and the merge does handle the bulkloader (but not some spreadsheet on your desktop, of course).

Where in the heck does "associated species" fall in all of this?

https://arctos.database.museum/tblbrowse.cfm?tbl=coll_object_remark

Is it part of the event stack or not?

No - but it should be moved to specimen_event, I think. Just a migration miss IIRC.

I can enter locality attributes as extras

You cannot: https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader

Why can't I enter event attributes as "extras"?

Maybe plausible, certainly open for discussion, a few could be flattened as well, I think I'd prefer to pre-create named events and just use them (but IDK how that aligns with reality).

[Jegelewicz]

I can enter locality attributes as extras

You cannot: https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader

UMMMM

[Jegelewicz]

Also

[dustymc]

"extras" - if you see it in the builder, it's not an extra.

[Jegelewicz]

Sorry for my poor use of "extras" - my argument is still the same, if I can do this for a locality, it seems that I should also be able to do it for an event, either directly or via "extras".

[Jegelewicz]

And finally - comparison for locality

Arctos Page/What it does	Higher Geography	Locality Name	Locality ID	Specific Locality	Minimum Elevation	Maximum Elevation	Elevation Units	Minimum Depth	Maximum Depth	Depth Units	Locality Remark	Coordinate Units	Decimal Latitiude	Decimal Longitude	Maximum Coordinate Error	Coordinate Error Units	Datum	Georeference Source	Georeference Protocol	WKT Media ID	Locality Attribute	Locality Attribute Value	Locality Attribute Units	Locality Attribute Determiner	Locality Attribute Determined Date	Locality Attribute Determined Method	Locality Attribute Remark
Edit catalog record	Higher Geography	Locality Nickname		Specific Locality	Min. Elev.	Max. Elev.	Elev. Unit	Min. Depth	Max. Depth	Depth Units	Locality Remarks	N/A	Decimal Latitude	Decimal Longitude	Max Error	Max Error Units	Datum	Georeference Source	Georeference Protocol	WKTMediaID	Type	Value	Units	Determiner	Date	Method	Remark
Catalog record	Higher Geography	Locality Nickname		Specific Locality	Elevation	(to) Elevation	Elevation	Depth	(to) Depth	Depth	Locality Remarks	N/A	Coordinates	Coordinates	Error	Error	Datum	Georeference Source	Georeference Protocol		Attribute	Value	Units	Determiner	Date	Method	Remark
Data entry (old)	Higher Geog	Locality Nickname	Existing LocalityID	Spec Locality	Elevation (min-max) between	Elevation (min-max) between	Elevation (min-max) between	Depth (min-max) between	Depth (min-max) between	Depth (min-max) between	LocalityRem	Original lat/long Units	Dec Lat	Dec Long	Max Error	Max Error	Datum	Georeference Source	Georeference Protocol	WKTMediaID	Type	Value	Unit	Determiner	Date	Method	Remark
Data entry (new)	Higher Geography	Locality Name	Locality ID	Specific Locality	Minimum Elevation	Maximum Elevation	Elevation Units	Minimum Depth	Maximum Depth	Depth Units	Locality Remarks	Coordinate Units	Dec Lat	Dec Long	Max Error	Max Error Units	Datum	Georeference Source	Georeference Protocol	WKTMediaID	locality_attribute_type	locality_attribute_value	locality_attribute_unit	locality_attribute_determiner	locality_attribute_detr_date	locality_attribute_detr_meth	locality_attribute_remark
Bulkloader builder	higher_geog	locality_name	locality_id	spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks	orig_lat_long_units	dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	wkt_media_id	locality_attribute_type	locality_attribute_value	locality_attribute_units	locality_attribute_determiner	locality_attribute_detr_date	locality_attribute_detr_meth	locality_attribute_remark
Bulkload specimen-events		locality_name		spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks	orig_lat_long_units	dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	wkt_media_id	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Bulkload collecting events	higher_geog			spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks		dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Bulkload locality
Bulkload locality attributes	N/A	locality_name	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	attribute_type	attribute_value	attribute_units	attribute_determiner	determined_date	determination_method	attribute_remark

I need to add in the names for bulkload locality - but the tool 404s for me right now. See #4053

A few things here.

1. No higher geography field in the specimen event loader
2. Datum is not included on the catalog record
3. I think we really should be recording who georeferenced and when they did it
4. When the georeferencing happens "behind the scenes" (coordinates other than decimal degrees are entered and used to create decimal degree values), the who would be "Arctos"?

[dustymc]

No higher geography field in the specimen event loader

>Datum is not included on the catalog record

> who georeferenced and when t

File a CT request

"behind the scenes" (coordinates other than decimal degrees are entered and used to create decimal degree values), the who would be "Arctos"?

There's only one possibility, we gain nothing by recording it.

[Jegelewicz]

No higher geography field in the specimen event loader

Get a template - there is no higher geography in it.

[Jegelewicz]

Datum is not included on the catalog record

Screen Shot 2021-10-28 at 4 31 54 PM

HMMMM I swear I saw one with coordinates but no datum - is it possible that it gets hidden when it is "unknown"? Guess not, I'm blaming that on late lunch...

[Jegelewicz]

A revised summary

Locality

Arctos Page/What it does	Higher Geography	Locality Name	Locality ID	Specific Locality	Minimum Elevation	Maximum Elevation	Elevation Units	Minimum Depth	Maximum Depth	Depth Units	Locality Remark	Coordinate Units	Decimal Latitiude	Decimal Longitude	Maximum Coordinate Error	Coordinate Error Units	Datum	Georeference Source	Georeference Protocol	WKT Media ID	Locality Attribute	Locality Attribute Value	Locality Attribute Units	Locality Attribute Determiner	Locality Attribute Determined Date	Locality Attribute Determined Method	Locality Attribute Remark
Edit catalog record	Higher Geography	Locality Nickname	N/A	Specific Locality	Min. Elev.	Max. Elev.	Elev. Unit	Min. Depth	Max. Depth	Depth Units	Locality Remarks	N/A	Decimal Latitude	Decimal Longitude	Max Error	Max Error Units	Datum	Georeference Source	Georeference Protocol	WKTMediaID	Type	Value	Units	Determiner	Date	Method	Remark
Catalog record	Higher Geography	Locality Nickname	N/A	Specific Locality	Elevation	(to) Elevation	Elevation	Depth	(to) Depth	Depth	Locality Remarks	N/A	Coordinates	Coordinates	Error	Error		Georeference Source	Georeference Protocol		Attribute	Value	Units	Determiner	Date	Method	Remark
Data entry (old)	Higher Geog	Locality Nickname	Existing LocalityID	Spec Locality	Elevation (min-max) between	Elevation (min-max) between	Elevation (min-max) between	Depth (min-max) between	Depth (min-max) between	Depth (min-max) between	LocalityRem	Original lat/long Units	Dec Lat	Dec Long	Max Error	Max Error	Datum	Georeference Source	Georeference Protocol	WKTMediaID	Type	Value	Unit	Determiner	Date	Method	Remark
Data entry (new)	Higher Geography	Locality Name	Locality ID	Specific Locality	Minimum Elevation	Maximum Elevation	Elevation Units	Minimum Depth	Maximum Depth	Depth Units	Locality Remarks	Coordinate Units	Dec Lat	Dec Long	Max Error	Max Error Units	Datum	Georeference Source	Georeference Protocol	WKTMediaID	locality_attribute_type	locality_attribute_value	locality_attribute_unit	locality_attribute_determiner	locality_attribute_detr_date	locality_attribute_detr_meth	locality_attribute_remark
Bulkloader builder	higher_geog	locality_name	locality_id	spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks	orig_lat_long_units	dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	wkt_media_id	locality_attribute_type	locality_attribute_value	locality_attribute_units	locality_attribute_determiner	locality_attribute_detr_date	locality_attribute_detr_meth	locality_attribute_remark
Bulkload specimen-events	higher_geog	locality_name	N/A	spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks	orig_lat_long_units	dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	wkt_media_id	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Bulkload collecting events	higher_geog	N/A	N/A	spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks	N/A	dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Bulkload locality	higher_geog	locality_name	N/A	spec_locality	minimum_elevation	maximum_elevation	orig_elev_units	min_depth	max_depth	depth_units	locality_remarks	N/A	dec_lat	dec_long	max_error_distance	max_error_units	datum	georeference_source	georeference_protocol	wkt_media_id	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Bulkload locality attributes	N/A	locality_name	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	attribute_type	attribute_value	attribute_units	attribute_determiner	determined_date	determination_method	attribute_remark
Search	Continent/Ocean, Sea, Country, State/Province, USGS Quad Map, County, Island Group, Island, Geographic Feature, Drainage	N/A	N/A	Specific Locality	Elevation	Elevation	Elevation	N/A	N/A	N/A	Locality Remarks	N/A	N/A	N/A	Coordinate Precision	Coordinate Precision	N/A	Georeference Source	N/A	N/A	Locality Attribute 1	Locality Attribute 1 Value	Locality Attribute 1 Unit	Locality Attribute 1 Determiner	N/A	Locality Attribute 1 Method	Locality Attribute 1 Remark

Event

Arctos Page/What it does	Verbatim Event Location	Verbatim Event Date	Formatted Event Start Date	Formatted Event End Date	Event Remark	Event Name	Verbatim Coordinates	Event Attribute Type	Event Attribute Value	Event Attribute Units	Event Attribute Determiner	Event Attribute Date	Event Attribute Method	Event Attribute Remark	Locality Name	Event ID	Locality ID
Edit catalog record	Verbatim Locality	Verbatim Date	Began Date/Time	Ended Date/Time	Collecting Event Remark	Collecting Event Nickname	As-entered Coordinates	Event Attribute Type	Event Attribute Value	Event Attribute Units	Event Attribute Determiner	Event Attribute Date	Event Attribute Method	Event Attribute Remark	locality_name	N/A	N/A
Catalog record	Verbatim Locality	Verbatim Date	Event Date	Event Date (to)	Remark	Event Nickname	As Entered	Attribute	Value	included in value if applicable	Determiner	Date	Method	Remark	Locality Nickname	N/A	N/A
Data entry (old)	Verbatim Locality	VerbatimDate	Begin	End	CollEvntRemk	Event Nickname	Coordinates (event and locality)	N/A	N/A	N/A	N/A	N/A	N/A	N/A	Locality Nickname	Existing EventID	N/A
Data entry (new)	Verbatim Locality	Verbatim Date	Began Date	Ended Date	Collecting Event Remark	Event Name	all the coordinate stuff that is not decimal degrees	N/A	N/A	N/A	N/A	N/A	N/A	N/A	Locality Name	Event ID	Locality ID
Bulkloader builder	verbatim_locality	verbatim_date	began_date	ended_date	coll_event_remarks	collecting_event_name	all the coordinate stuff that is not decimal degrees	N/A	N/A	N/A	N/A	N/A	N/A	N/A	locality_name	collecting_event_id	locality_id
Bulkload specimen-events	verbatim_locality	verbatim_date	began_date	ended_date	coll_event_remarks	collecting_event_name	all the coordinate stuff that is not decimal degrees	N/A	N/A	N/A	N/A	N/A	N/A	N/A	locality_name	N/A	N/A
Bulkload collecting events	verbatim_locality	verbatim_date	began_date	ended_date	coll_event_remarks	collecting_event_name	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	locality_name	N/A	N/A
Bulkload collecting event attributes	N/A	N/A	N/A	N/A	N/A	event_name	N/A	event_attribute_type	event_attribute_value	event_attribute_units	event_determiner	event_determined_date	event_determination_method	event_attribute_remark	N/A	N/A	N/A
Search	Verbatim Locality	Verbatim Date	Collected On or After	Collected On or Before	Collecting Event Remark	Search	N/A	Event Attributes	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A

Event Assignment

Arctos Page/What it does	Assigned Event Type	Event Assigned By	Event Assigned Date	Event Verification	Event Verified By	Event Verification Date	Event Assignment Remark	Event Habitat	Event Source	Event Method
Edit catalog record	Specimen/Event Type	Event Determiner	Determined Date	Verification Status	Verified By	Verified Date	Specimen/Event Remark	Habitat	Collecting Source	Collecting Method
Catalog record	Event Type	Event Type: assigned by	Event Date	Event Verification:	Event Verification: verified by	Event Verification: on	Event Details: remark	Event Details: habitat	Event Details: Collecting Source	Event Details: collecting method
Data entry (old)	Specimen/Event Type	Event Determiner	Detr. Date	VerificationStatus	N/A	N/A	Specimen/Event Remark	Habitat	Coll. Src.	Coll. Meth.
Data entry (new)	Event Type	Event Determiner	Det. Date	VerificationStatus	N/A	N/A	Event Remark	Habitat	Collecting Source	Collecting Method
Bulkloader builder	specimen_event_type	event_assigned_by_agent	event_assigned_date	verificationstatus	N/A	N/A	specimen_event_remark	habitat	collecting_source	collecting_method
Bulkload specimen-events	specimen_event_type	assigned_by_agent	assigned_date	verificationstatus	N/A	N/A	specimen_event_remark	habitat	collecting_source	collecting_method
Search	Record/Event Type	N/A	N/A	Verification Status	N/A	N/A	Record/Event Remark	Habitat * in Locality	Collecting Source	Collecting Method