Custom Z segments at different levels in ORU^R01 messages

[dpfeif]

Hi,

First of all, thank you for making this library publicly available and maintaining it 👍

I'm testing out some scenarios and I came across a situation that I can't quite understand. I was hoping you could help shed some light on what I'm doing wrong / what's going wrong.

"""
    Test Z segments in ORU-R01 messages
    hl7apy==1.3.4
"""

from hl7apy.parser import parse_message

er7 = """
MSH|^~\&|1101|PAML|7000|DrSmith|20130909133853|-5d4a2583:140c1764186:-255e|ORU^R01^ORU_R01|-5d4a2583:140c1764186:-255e|P|2.5|
PID|1||0123456789||PATIENT01^TEST||19700101|M||ASA|1234 Here Street^^Spokane^WA^99202||(509)555-1212^^^^^509^5551212|||||EEPATIENT01
PV1|1||^^^DrSmith^^^Building1^^Description||||1234567890^DR SMITH||||||||||||||||||||||||||||||||654321|
ORC|RE|123456|||||||20130607103421|
OBR|1|123456|M1026954|80100.DA33^DAT Apolipoprotein A|R||201309091313|||||||||1234567890^DR SMITH|||C||||||F
OBX|1|NM|1869-7^Apolipoprotein A 1||154|mg/dL|79-169||||F||||50D0661605^PAML, 110 W. Cliff Dr, Spokane, WA 99204
""".replace("\n", "\r")

m = parse_message(er7)

# Make the message valid
m.oru_r01_patient_result.oru_r01_patient.oru_r01_visit.pv1.pv1_2  = "TEST"

# add my cutstom segments
m.zin.zin_2 = "My custom test z segment"
m.oru_r01_patient_result.oru_r01_patient.zpc.zpc_2 = "Another custom z segment"
m.oru_r01_patient_result.oru_r01_order_observation.zor.zor_2 = "Yet another custom z segment"
m.oru_r01_patient_result.oru_r01_order_observation.oru_r01_observation.zox.zox_2 = "Yet again another custom z segment"

print(f"Validation: {m.validate()}")
print(f"ER7:\n{m.to_er7()}\n")
print(f"Message children: {m.children}")
print(f"ORU_R01_PATIENT children: {m.oru_r01_patient_result.oru_r01_patient.children}")
print(f"ORU_R01_ORDER_OBSERVATION children: {m.oru_r01_patient_result.oru_r01_order_observation.children}")
print(f"ORU_R01_OBSERVATION children: {m.oru_r01_patient_result.oru_r01_order_observation.oru_r01_observation.children}")
print("")

reparsed_message = parse_message(m.to_er7())
try:
    reparsed_validation = reparsed_message.validate()
except Exception as e:
    reparsed_validation = e

print(f"Validation after reparsing: {reparsed_validation}")
print(f"ER7 after reparsing:\n{reparsed_message.to_er7()}\n")
print(f"Message children after reparsing: {reparsed_message.children}")
print(f"ORU_R01_PATIENT children after reparsing: {reparsed_message.oru_r01_patient_result.oru_r01_patient.children}")
print(f"ORU_R01_ORDER_OBSERVATION children after reparsing: {reparsed_message.oru_r01_patient_result.oru_r01_order_observation.children}")
print(f"ORU_R01_OBSERVATION children after reparsing: {reparsed_message.oru_r01_patient_result.oru_r01_order_observation.oru_r01_observation.children}")

which outputs:

Validation: True
ER7:
MSH|^~\&|1101|PAML|7000|DrSmith|20130909133853|-5d4a2583:140c1764186:-255e|ORU^R01^ORU_R01|-5d4a2583:140c1764186:-255e|PPID|1||0123456789||PATIENT01^TEST||19700101|M||ASA|1234 Here Street^^Spokane^WA^99202||(509)555-1212^^^^^509^5551212||||ZIN||My custom test z fieldield field54|mg/dL|79-169||||F||||50D0661605^PAML, 110 W. Cliff Dr, Spokane, WA 99204

Message children: [<Segment MSH>, <Group ORU_R01_PATIENT_RESULT>, <Segment ZIN>]
ORU_R01_PATIENT children: [<Segment PID>, <Group ORU_R01_VISIT>, <Segment ZPC>]
ORU_R01_ORDER_OBSERVATION children: [<Segment ORC>, <Segment OBR>, <Group ORU_R01_OBSERVATION>, <Segment ZOR>]
ORU_R01_OBSERVATION children: [<Segment OBX>, <Segment ZOX>]

Validation after reparsing: Missing required child ORU_R01_PATIENT_RESULT.ORU_R01_ORDER_OBSERVATION
ER7 after reparsing:
MSH|^~\&|1101|PAML|7000|DrSmith|20130909133853|-5d4a2583:140c1764186:-255e|ORU^R01^ORU_R01|-5d4a2583:140c1764186:-255e|PPID|1||0123456789||PATIENT01^TEST||19700101|M||ASA|1234 Here Street^^Spokane^WA^99202||(509)555-1212^^^^^509^5551212||||OBX|1|NM|1869-7^Apolipoprotein A 1||154|mg/dL|79-169||||F||||50D0661605^PAML, 110 W. Cliff Dr, Spokane, WA 99204

Message children after reparsing: [<Segment MSH>, <Group ORU_R01_PATIENT_RESULT>, <Group ORU_R01_PATIENT_RESULT>]
ORU_R01_PATIENT children after reparsing: [<Segment PID>, <Group ORU_R01_VISIT>]
ORU_R01_ORDER_OBSERVATION children after reparsing: []
ORU_R01_OBSERVATION children after reparsing: []

I was expecting to be able to keep my custom fields across ER7 marshaling and unmarshaling. Also, I did not expect the final result to have 2 patient result groups.

Best regards,

[svituz]

Hi @dpfeif,
you're right, it's very strange behavior. I need to investigate it more deeply.

[monkeyden]

I found this issue while investigating my own occurrence. After some investigation, I believe this is the correct behavior of the parser. Segment grouping is on by default when parsing:

def parse_message(message, validation_level=None, find_groups=True, message_profile=None, report_file=None, force_validation=False):

Because custom Z segments aren't part of any group they don't appear in the parsed message. Setting find_groups=False should yield the segments in the parsed message for you.