Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The import has been aborted because the entry does not have a parent entry #438

Closed
JesseCoretta opened this issue Oct 18, 2024 · 21 comments · Fixed by #444
Closed

The import has been aborted because the entry does not have a parent entry #438

JesseCoretta opened this issue Oct 18, 2024 · 21 comments · Fixed by #444
Assignees

Comments

@JesseCoretta
Copy link
Contributor

JesseCoretta commented Oct 18, 2024

Describe the bug

Re-emergence of #431

On a fresh backend, I am once again seeing

[17/Oct/2024:20:06:00 -0700] category=BACKEND severity=ERROR msgID=-1 msg=failed to execute task exception=ExecutionException: org.opends.server.types.DirectoryException: The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry (FutureTask.java:122 FutureTask.java:191 OnDiskMergeImporter.java:2798 OnDiskMergeImporter.java:1281 OnDiskMergeImporter.java:1040 OnDiskMergeImporter.java:154 OnDiskMergeImporter.java:228 BackendImpl.java:660 ImportLDIF.java:858 TaskTool.java:251 ImportLDIF.java:243 ImportLDIF.java:113 ImportLDIF.java:88)
[17/Oct/2024:20:06:00 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/entryUUID.uuidMatch phase two processing completed
[17/Oct/2024:20:06:00 -0700] category=BACKEND severity=NOTICE msgID=526 msg=Import LDIF environment close took 0 seconds
[17/Oct/2024:20:06:00 -0700] category=BACKEND severity=NOTICE msgID=518 msg=Flushing data to disk
[17/Oct/2024:20:06:00 -0700] category=BACKEND severity=ERROR msgID=481 msg=The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry

Commands run:

$ sudo /opt/opendj/bin/dsconfig create-backend \
  -D "cn=Registration Authority" \
  -w secret \
  --backend-name=rA \
  -t je \
  --set=base-dn:o=rA \
  --set=enabled:true \
  --set=index-entry-limit:200000 \
  --no-prompt --trustAll

$ sudo /opt/opendj/bin/stop-ds

$ sudo /opt/opendj/bin/import-ldif \
  -D "cn=Registration Authority" \
  -w secret \
  --clearBackend \
  --backendID=rA \
  --ldifFile ~/dev/tmp/content.ldif \
  --offline

Note, I also tried without --clearBackend with the same result.

@vharseko vharseko changed the title LDIF Loader still not functioning The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry Oct 18, 2024
@vharseko
Copy link
Member

please check "exists parent entry for 'n=0,ou=Registrations,o=rA' in ~/dev/tmp/content.ldif

@JesseCoretta
Copy link
Contributor Author

This is the same LDIF I was using in my original ticket @vharseko. But, nonetheless, the following is the very top of the file:

dn: o=rA                                                                
objectClass: organization                                               
                                                                        
dn: ou=Registrations,o=rA                                               
objectClass: organizationalUnit                                         
             
dn: n=0,ou=Registrations,o=rA                                           
objectClass: top                                                        
objectClass: registration                                               
objectClass: rootArc                                                    
objectClass: x660Context                                                
objectClass: x680Context                                                
unicodeValue: ITU-T                                                     
n: 0                                                                    
aSN1Notation: {itu-t(0)}                                                
identifier: itu-t                                                       
nameAndNumberForm: itu-t(0)                                             
iRI: /ITU-T 

Procedure:

$ sudo /opt/opendj/bin/dsconfig \
  delete-backend \
  --bindDN="cn=Registration Authority" \
  --bindPassword=secret \
  --backend-name=rA \
  --no-prompt && sudo rm -rf /opt/opendj/db/rA
$ sudo /opt/opendj/bin/dsconfig \
  create-backend \
  -D "cn=Registration Authority" \
  -w secret \
  --backend-name=rA \
  -t je \
  --set=base-dn:o=rA \
  --set=enabled:true \
  --no-prompt --trustAll

$ sudo /opt/opendj/bin/stop-ds

$ sudo /opt/opendj/bin/import-ldif \
  -D "cn=Registration Authority" \
  -w secret \
  --backendID=rA \
  --ldifFile ~/dev/tmp/content.ldif \
  --offline

Load output:

[18/Oct/2024:00:36:03 -0700] category=JVM severity=NOTICE msgID=21 msg=Installation Directory:  /opt/opendj
[18/Oct/2024:00:36:03 -0700] category=JVM severity=NOTICE msgID=23 msg=Instance Directory:      /opt/opendj
[18/Oct/2024:00:36:03 -0700] category=JVM severity=NOTICE msgID=17 msg=JVM Information: 11.0.24+8-post-Ubuntu-1ubuntu322.04 by Ubuntu, 64-bit architecture, 4152360960 bytes heap size
[18/Oct/2024:00:36:03 -0700] category=JVM severity=NOTICE msgID=18 msg=JVM Host: hplptp, running Linux 6.8.0-45-generic amd64, 16607137792 bytes physical memory size, number of processors available 8
[18/Oct/2024:00:36:03 -0700] category=JVM severity=NOTICE msgID=19 msg=JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED", "-Dorg.opends.server.scriptName=import-ldif"
[18/Oct/2024:00:36:03 -0700] category=BACKEND severity=INFORMATION msgID=410 msg=JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads
[18/Oct/2024:00:36:03 -0700] category=BACKEND severity=INFORMATION msgID=411 msg=JE backend 'rA' does not specify the number of lock tables: defaulting to 17
[18/Oct/2024:00:36:03 -0700] category=BACKEND severity=NOTICE msgID=533 msg=OpenDJ Server 4.8.1 starting import (build 20241017142849, R974cbbeaa39873742bdfdaca8d6f3ddb868cf36c)
[18/Oct/2024:00:36:03 -0700] category=BACKEND severity=NOTICE msgID=528 msg=The amount of free memory available to the import task is 3737124864 bytes. The number of phase one buffers required is 112 buffers
[18/Oct/2024:00:36:03 -0700] category=BACKEND severity=NOTICE msgID=520 msg=Setting DB cache size to 33554432 bytes and phase one buffer size to 10010155 bytes
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=534 msg=Import Thread Count: 8 threads
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/dn2id phase two started processing 1 buffers in 1 batches
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two started processing 1 buffers in 1 batches
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/entryUUID.uuidMatch phase two started processing 1 buffers in 1 batches
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/entryUUID.uuidMatch phase two processing completed
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two processing completed
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/dn2id phase two processing completed
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=ERROR msgID=-1 msg=failed to execute task exception=ExecutionException: org.opends.server.types.DirectoryException: The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry (FutureTask.java:122 FutureTask.java:191 OnDiskMergeImporter.java:2798 OnDiskMergeImporter.java:1281 OnDiskMergeImporter.java:1040 OnDiskMergeImporter.java:154 OnDiskMergeImporter.java:228 BackendImpl.java:660 ImportLDIF.java:858 TaskTool.java:251 ImportLDIF.java:243 ImportLDIF.java:113 ImportLDIF.java:88)
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=526 msg=Import LDIF environment close took 0 seconds
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=NOTICE msgID=518 msg=Flushing data to disk
[18/Oct/2024:00:36:04 -0700] category=BACKEND severity=ERROR msgID=481 msg=The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry

Yet, it most certainly DOES have a parent (see the top of this reply). But, to prove the point even more:

# search the DIT to see what made it in ...

$ ldapsearch -w secret -D 'cn=Registration Authority' -xb o=rA -s sub

# extended LDIF                                                         
#                                                                       
# LDAPv3                                                                
# base <o=rA> with scope subtree                                        
# filter: (objectclass=*)                                               
# requesting: ALL                                                       
#                                                                       
                                                                        
# rA                                                                    
dn: o=rA                                                                
objectClass: organization                                               
objectClass: top                                                        
o: rA                                                                   
                                                                        
# Registrations, rA                                                     
dn: ou=Registrations,o=rA                                               
objectClass: top                                                        
objectClass: organizationalUnit                                         
ou: Registrations                                                       
                                                                        
# search result                                                         
search: 2                                                               
result: 0 Success                                                       
                                                                        
# numResponses: 3                                                       
# numEntries: 2

The parent did, in fact, make it to the DB ...

Again, ldapadd works:

$ ldapadd -c -D "cn=Registration Authority" -w secret -xf content.ldif 

adding new entry "n=0,ou=Registrations,o=rA"

... and confirm again by ldapsearch:

# same command we just ran ...

$ ldapsearch -w secret -D 'cn=Registration Authority' -xb o=rA -s sub

# extended LDIF                                                         
#                                                                       
# LDAPv3                                                                
# base <o=rA> with scope subtree                                        
# filter: (objectclass=*)                                               
# requesting: ALL                                                       
#                                                                       
                                                                        
# rA                                                                    
dn: o=rA                                                                
objectClass: organization                                               
objectClass: top                                                        
o: rA                                                                   
                                                                        
# Registrations, rA                                                     
dn: ou=Registrations,o=rA                                               
objectClass: top                                                        
objectClass: organizationalUnit                                         
ou: Registrations                                                       
                                                                        
# 0, Registrations, rA                                                  
dn: n=0,ou=Registrations,o=rA                                           
objectClass: x680Context                                                
objectClass: x660Context                                                
objectClass: top                                                        
objectClass: rootArc                                                    
objectClass: registration                                               
identifier: itu-t                                                       
aSN1Notation: {itu-t(0)}                                                
unicodeValue: ITU-T                                                     
iRI: /ITU-T                                                             
nameAndNumberForm: itu-t(0)                                             
n: 0

@vharseko
Copy link
Member

try option --threadCount 1 from import-ldif

Number of threads used to read LDIF file during import. Default value (0) equals: 2 x (number of CPUs).

@JesseCoretta
Copy link
Contributor Author

Unfortunately, no joy. See the diagnostics below, and see some WEIRD results at the very end.

$ sudo /opt/opendj/bin/dsconfig delete-backend --bindDN="cn=Registration Authority" --bindPassword=secret --backend-name=rA --no-prompt && sudo rm -rf /opt/opendj/db/rA

The Backend was deleted successfully
$ sudo /opt/opendj/bin/dsconfig create-backend -D "cn=Registration Authority" -w secret --backend-name=rA -t je --set=base-dn:o=rA --set=enabled:true --no-prompt --trustAll
$ sudo /opt/opendj/bin/stop-ds

Stopping Server...
$ sudo /opt/opendj/bin/import-ldif -D "cn=Registration Authority" -w secret  --backendID=rA --ldifFile ~/dev/tmp/content.ldif --threadCount=1 --offline

Log output (note: 14th log event confirms thread count == 1)

[18/Oct/2024:01:02:23 -0700] category=BACKEND severity=NOTICE msgID=370 msg=The backend userRoot is now taken offline
[18/Oct/2024:01:02:23 -0700] category=BACKEND severity=NOTICE msgID=370 msg=The backend rA is now taken offline
[18/Oct/2024:01:02:23 -0700] category=CORE severity=NOTICE msgID=203 msg=The Directory Server is now stopped
[18/Oct/2024:01:02:26 -0700] category=JVM severity=NOTICE msgID=21 msg=Installation Directory:  /opt/opendj
[18/Oct/2024:01:02:26 -0700] category=JVM severity=NOTICE msgID=23 msg=Instance Directory:      /opt/opendj
[18/Oct/2024:01:02:26 -0700] category=JVM severity=NOTICE msgID=17 msg=JVM Information: 11.0.24+8-post-Ubuntu-1ubuntu322.04 by Ubuntu, 64-bit architecture, 4152360960 bytes heap size
[18/Oct/2024:01:02:26 -0700] category=JVM severity=NOTICE msgID=18 msg=JVM Host: hplptp, running Linux 6.8.0-45-generic amd64, 16607137792 bytes physical memory size, number of processors available 8
[18/Oct/2024:01:02:26 -0700] category=JVM severity=NOTICE msgID=19 msg=JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED", "-Dorg.opends.server.scriptName=import-ldif"
[18/Oct/2024:01:02:26 -0700] category=BACKEND severity=INFORMATION msgID=410 msg=JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads
[18/Oct/2024:01:02:26 -0700] category=BACKEND severity=INFORMATION msgID=411 msg=JE backend 'rA' does not specify the number of lock tables: defaulting to 17
[18/Oct/2024:01:02:26 -0700] category=BACKEND severity=NOTICE msgID=533 msg=OpenDJ Server 4.8.1 starting import (build 20241017142849, R974cbbeaa39873742bdfdaca8d6f3ddb868cf36c)
[18/Oct/2024:01:02:26 -0700] category=BACKEND severity=NOTICE msgID=528 msg=The amount of free memory available to the import task is 3737124864 bytes. The number of phase one buffers required is 14 buffers
[18/Oct/2024:01:02:26 -0700] category=BACKEND severity=NOTICE msgID=520 msg=Setting DB cache size to 33554432 bytes and phase one buffer size to 80081247 bytes
[18/Oct/2024:01:02:27 -0700] category=BACKEND severity=NOTICE msgID=534 msg=Import Thread Count: 1 threads
[18/Oct/2024:01:02:33 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two started processing 1 buffers in 1 batches
[18/Oct/2024:01:02:33 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two processing completed
[18/Oct/2024:01:02:33 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/entryUUID.uuidMatch phase two started processing 1 buffers in 1 batches
[18/Oct/2024:01:02:33 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/dn2id phase two started processing 1 buffers in 1 batches
[18/Oct/2024:01:02:34 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/dn2id phase two processing completed
[18/Oct/2024:01:02:34 -0700] category=BACKEND severity=ERROR msgID=-1 msg=failed to execute task exception=ExecutionException: org.opends.server.types.DirectoryException: The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry (FutureTask.java:122 FutureTask.java:191 OnDiskMergeImporter.java:2798 OnDiskMergeImporter.java:1281 OnDiskMergeImporter.java:1040 OnDiskMergeImporter.java:154 OnDiskMergeImporter.java:228 BackendImpl.java:660 ImportLDIF.java:858 TaskTool.java:251 ImportLDIF.java:243 ImportLDIF.java:113 ImportLDIF.java:88)
[18/Oct/2024:01:02:34 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/entryUUID.uuidMatch phase two processing completed
[18/Oct/2024:01:02:34 -0700] category=BACKEND severity=NOTICE msgID=526 msg=Import LDIF environment close took 0 seconds
[18/Oct/2024:01:02:34 -0700] category=BACKEND severity=NOTICE msgID=518 msg=Flushing data to disk
[18/Oct/2024:01:02:34 -0700] category=BACKEND severity=ERROR msgID=481 msg=The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry

Start server and do ldapsearch:

$ sudo /opt/opendj/bin/start-ds
...
[18/Oct/2024:01:05:43 -0700] category=BACKEND severity=NOTICE msgID=513 msg=The database backend rA containing 0 entries has started
...

So, in case you didn't see above, it says zero (0) entries in rA, so naturally my ldapsearch should show zero (0) entries. Right?

$ ldapsearch -w secret -D 'cn=Registration Authority' -xb o=rA -s sub 

.... LOTS of content omitted ...

# Registrations, rA                                                     
dn: ou=Registrations,o=rA                                               
objectClass: top                                                        
objectClass: organizationalUnit                                         
ou: Registrations                                                       
                                                                        
# search result                                                         
search: 2                                                               
result: 0 Success                                                       
                                                                        
# numResponses: 62621                                                   
# numEntries: 62620

WOW: 0 < 62620

In looking at the entries, I notice they're ALL registrant STRUCTURAL entries besides the OUs and suffix root.

No registration instances were loaded. I searched the complete output -- not a single child of ou=Registrations,o=rA has loaded.

For your reference, I'm going to identify the KINDs of each class, and the syntax of each attribute. I have no idea why import-ldif would behave differently than ldapadd with regards to such nuances ... but maybe it will spark an idea:

dn: n=0,ou=Registrations,o=rA                                           
objectClass: top                # ABSTRACT
objectClass: registration       # ABSTRACT (top is SUP)
objectClass: rootArc            # STRUCTURAL (registration is SUP)
objectClass: x660Context        # AUXILIARY (registration is SUP)
objectClass: x680Context        # AUXILIARY (registration is SUP)
unicodeValue: ITU-T             # OCTET STRING
n: 0                            # INTEGER
aSN1Notation: {itu-t(0)}        # DIRECTORY STRING
identifier: itu-t               # SUP: NAME
nameAndNumberForm: itu-t(0)     # DIRECTORY STRING
iRI: /ITU-T                     # OCTET STRING

@vharseko
Copy link
Member

please attach test ldif for reproduce

@JesseCoretta
Copy link
Contributor Author

I can only give you an excerpt, because many email addresses are in the file. Even if they're already publicized, I'd rather not post them here. 🤐

FORTUNATELY, even the small excerpt triggers the issue. See attached (note I had to rename it to include ".txt" as ".ldif" is not allowed). I even threw a subentry in there, just because they're present as well as custom entries.

content.ldif.txt

I also attached the schema file again, just in case.

And, just to show you the excerpt triggers the same issue on a freshly deleted/created backend:

$ sudo /opt/opendj/bin/import-ldif \
  -D "cn=Registration Authority" \
  -w secret \
  --backendID=rA \
  --threadCount=1 \
  --ldifFile ~/dev/tmp/content.ldif \
  --offline

[18/Oct/2024:01:38:48 -0700] category=JVM severity=NOTICE msgID=21 msg=Installation Directory:  /opt/opendj
[18/Oct/2024:01:38:48 -0700] category=JVM severity=NOTICE msgID=23 msg=Instance Directory:      /opt/opendj
[18/Oct/2024:01:38:48 -0700] category=JVM severity=NOTICE msgID=17 msg=JVM Information: 11.0.24+8-post-Ubuntu-1ubuntu322.04 by Ubuntu, 64-bit architecture, 4152360960 bytes heap size
[18/Oct/2024:01:38:48 -0700] category=JVM severity=NOTICE msgID=18 msg=JVM Host: hplptp, running Linux 6.8.0-45-generic amd64, 16607137792 bytes physical memory size, number of processors available 8
[18/Oct/2024:01:38:48 -0700] category=JVM severity=NOTICE msgID=19 msg=JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED", "-Dorg.opends.server.scriptName=import-ldif"
[18/Oct/2024:01:38:48 -0700] category=BACKEND severity=INFORMATION msgID=410 msg=JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads
[18/Oct/2024:01:38:48 -0700] category=BACKEND severity=INFORMATION msgID=411 msg=JE backend 'rA' does not specify the number of lock tables: defaulting to 17
[18/Oct/2024:01:38:49 -0700] category=BACKEND severity=NOTICE msgID=533 msg=OpenDJ Server 4.8.1 starting import (build 20241017142849, R974cbbeaa39873742bdfdaca8d6f3ddb868cf36c)
[18/Oct/2024:01:38:49 -0700] category=BACKEND severity=NOTICE msgID=528 msg=The amount of free memory available to the import task is 3737124864 bytes. The number of phase one buffers required is 112 buffers
[18/Oct/2024:01:38:49 -0700] category=BACKEND severity=NOTICE msgID=520 msg=Setting DB cache size to 33554432 bytes and phase one buffer size to 10010155 bytes
[18/Oct/2024:01:38:49 -0700] category=BACKEND severity=NOTICE msgID=534 msg=Import Thread Count: 8 threads
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/dn2id phase two started processing 1 buffers in 1 batches
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/entryUUID.uuidMatch phase two started processing 1 buffers in 1 batches
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two started processing 1 buffers in 1 batches
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/entryUUID.uuidMatch phase two processing completed
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two processing completed
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/dn2id phase two processing completed
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=ERROR msgID=-1 msg=failed to execute task exception=ExecutionException: org.opends.server.types.DirectoryException: The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry (FutureTask.java:122 FutureTask.java:191 OnDiskMergeImporter.java:2798 OnDiskMergeImporter.java:1281 OnDiskMergeImporter.java:1040 OnDiskMergeImporter.java:154 OnDiskMergeImporter.java:228 BackendImpl.java:660 ImportLDIF.java:858 TaskTool.java:251 ImportLDIF.java:243 ImportLDIF.java:113 ImportLDIF.java:88)
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=526 msg=Import LDIF environment close took 0 seconds
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=NOTICE msgID=518 msg=Flushing data to disk
[18/Oct/2024:01:38:50 -0700] category=BACKEND severity=ERROR msgID=481 msg=The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry

@vharseko
Copy link
Member

online import works done ?

@JesseCoretta
Copy link
Contributor Author

Online method does not seem to work:

## Delete backend
$ sudo /opt/opendj/bin/dsconfig delete-backend --bindDN="cn=Registration Authority" --bindPassword=secret --backend-name=rA --no-prompt && sudo rm -rf /opt/opendj/db/rA

The Backend was deleted successfully

## Create backend
$ sudo /opt/opendj/bin/dsconfig create-backend -D "cn=Registration Authority" -w secret --backend-name=rA -t je --set=base-dn:o=rA --set=enabled:true --no-prompt --trustAll

Online import:

$ sudo /opt/opendj/bin/import-ldif -D "cn=Registration Authority" -w secret --backendID=rA --ldifFile ~/dev/tmp/content.ldif 

Import task 20241018051030638 scheduled to start immediately
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=0 msgID=org.opends.messages.backend-413 message="Import task 20241018051030638 started execution"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=1 msgID=org.opends.messages.backend-370 message="The backend rA is now taken offline"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=2 msgID=org.opends.messages.runtime-21 message="Installation Directory:  /opt/opendj"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=3 msgID=org.opends.messages.runtime-23 message="Instance Directory:      /opt/opendj"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=4 msgID=org.opends.messages.runtime-17 message="JVM Information: 11.0.24+8-post-Ubuntu-1ubuntu322.04 by Ubuntu, 64-bit architecture, 4152360960 bytes heap size"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=5 msgID=org.opends.messages.runtime-18 message="JVM Host: hplptp, running Linux 6.8.0-45-generic amd64, 16607137792 bytes physical memory size, number of processors available 8"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=6 msgID=org.opends.messages.runtime-19 message="JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED", "-Dorg.opends.server.scriptName=start-ds""
[18/Oct/2024:05:10:30 -0700] severity="INFORMATION" msgCount=7 msgID=org.opends.messages.backend-410 message="JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads"
[18/Oct/2024:05:10:30 -0700] severity="INFORMATION" msgCount=8 msgID=org.opends.messages.backend-411 message="JE backend 'rA' does not specify the number of lock tables: defaulting to 17"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=9 msgID=org.opends.messages.backend-533 message="OpenDJ Server 4.8.1 starting import (build 20241017142849, R974cbbeaa39873742bdfdaca8d6f3ddb868cf36c)"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=10 msgID=org.opends.messages.backend-528 message="The amount of free memory available to the import task is 3737124864 bytes. The number of phase one buffers required is 56 buffers"
[18/Oct/2024:05:10:30 -0700] severity="NOTICE" msgCount=11 msgID=org.opends.messages.backend-520 message="Setting DB cache size to 33554432 bytes and phase one buffer size to 20020311 bytes"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=12 msgID=org.opends.messages.backend-534 message="Import Thread Count: 4 threads"
[18/Oct/2024:05:10:31 -0700] severity="ERROR" msgCount=13 msgID=null--1 message="failed to execute task" exception="ExecutionException: org.opends.server.types.DirectoryException: The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry (FutureTask.java:122 FutureTask.java:191 OnDiskMergeImporter.java:2798 OnDiskMergeImporter.java:1281 OnDiskMergeImporter.java:1040 OnDiskMergeImporter.java:154 OnDiskMergeImporter.java:228 BackendImpl.java:660 ImportTask.java:648 Task.java:965 TaskThread.java:179)"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=14 msgID=org.opends.messages.backend-526 message="Import LDIF environment close took 0 seconds"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=15 msgID=org.opends.messages.backend-518 message="Flushing data to disk"
[18/Oct/2024:05:10:31 -0700] severity="ERROR" msgCount=16 msgID=org.opends.messages.backend-481 message="The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry"
[18/Oct/2024:05:10:31 -0700] severity="INFORMATION" msgCount=17 msgID=org.opends.messages.backend-410 message="JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads"
[18/Oct/2024:05:10:31 -0700] severity="INFORMATION" msgCount=18 msgID=org.opends.messages.backend-411 message="JE backend 'rA' does not specify the number of lock tables: defaulting to 17"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=19 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_aci is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=20 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_entryUUID is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=21 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_objectClass is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=22 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_ds-sync-hist is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=23 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_ds-sync-conflict is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=24 msgID=org.opends.messages.backend-513 message="The database backend rA containing 0 entries has started"
[18/Oct/2024:05:10:31 -0700] severity="NOTICE" msgCount=25 msgID=org.opends.messages.backend-414 message="Import task 20241018051030638 finished execution in the state Stopped by error"
Import task 20241018051030638 did not complete successfully

As you can see, in addition to the "no parent" issue, it is also complaining about degraded indices, which it shouldn't because the backend was just initialized. I don't see that behavior in --offline mode.

@vharseko
Copy link
Member

try with -S

-S | --skipSchemaValidation
Skip schema validation during the LDIF import.
Default: false

@JesseCoretta
Copy link
Contributor Author

JesseCoretta commented Oct 18, 2024

Exact same result as without -S:

$ sudo /opt/opendj/bin/dsconfig delete-backend --bindDN="cn=Registration Authority" --bindPassword=secret --backend-name=rA --no-prompt && sudo rm -rf /opt/opendj/db/rA

The Backend was deleted successfully

$ sudo /opt/opendj/bin/dsconfig create-backend -D "cn=Registration Authority" -w secret --backend-name=rA -t je --set=base-dn:o=rA --set=enabled:true --no-prompt --trustAll
$ sudo /opt/opendj/bin/import-ldif -D "cn=Registration Authority" -w secret --backendID=rA --threadCount=1 -S --ldifFile ~/dev/tmp/content.ldif

Import task 20241018130645873 scheduled to start immediately
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=0 msgID=org.opends.messages.backend-413 message="Import task 20241018130645873 started execution"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=1 msgID=org.opends.messages.backend-370 message="The backend rA is now taken offline"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=2 msgID=org.opends.messages.runtime-21 message="Installation Directory:  /opt/opendj"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=3 msgID=org.opends.messages.runtime-23 message="Instance Directory:      /opt/opendj"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=4 msgID=org.opends.messages.runtime-17 message="JVM Information: 11.0.24+8-post-Ubuntu-1ubuntu322.04 by Ubuntu, 64-bit architecture, 4152360960 bytes heap size"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=5 msgID=org.opends.messages.runtime-18 message="JVM Host: hplptp, running Linux 6.8.0-47-generic amd64, 16607141888 bytes physical memory size, number of processors available 8"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=6 msgID=org.opends.messages.runtime-19 message="JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED", "-Dorg.opends.server.scriptName=start-ds""
[18/Oct/2024:13:06:45 -0700] severity="INFORMATION" msgCount=7 msgID=org.opends.messages.backend-410 message="JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads"
[18/Oct/2024:13:06:45 -0700] severity="INFORMATION" msgCount=8 msgID=org.opends.messages.backend-411 message="JE backend 'rA' does not specify the number of lock tables: defaulting to 17"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=9 msgID=org.opends.messages.backend-533 message="OpenDJ Server 4.8.1 starting import (build 20241017142849, R974cbbeaa39873742bdfdaca8d6f3ddb868cf36c)"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=10 msgID=org.opends.messages.backend-528 message="The amount of free memory available to the import task is 3737124864 bytes. The number of phase one buffers required is 14 buffers"
[18/Oct/2024:13:06:45 -0700] severity="NOTICE" msgCount=11 msgID=org.opends.messages.backend-520 message="Setting DB cache size to 33554432 bytes and phase one buffer size to 80081247 bytes"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=12 msgID=org.opends.messages.backend-534 message="Import Thread Count: 1 threads"
[18/Oct/2024:13:06:46 -0700] severity="ERROR" msgCount=13 msgID=null--1 message="failed to execute task" exception="ExecutionException: org.opends.server.types.DirectoryException: The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry (FutureTask.java:122 FutureTask.java:191 OnDiskMergeImporter.java:2798 OnDiskMergeImporter.java:1281 OnDiskMergeImporter.java:1040 OnDiskMergeImporter.java:154 OnDiskMergeImporter.java:228 BackendImpl.java:660 ImportTask.java:648 Task.java:965 TaskThread.java:179)"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=14 msgID=org.opends.messages.backend-526 message="Import LDIF environment close took 0 seconds"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=15 msgID=org.opends.messages.backend-518 message="Flushing data to disk"
[18/Oct/2024:13:06:46 -0700] severity="ERROR" msgCount=16 msgID=org.opends.messages.backend-481 message="The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry"
[18/Oct/2024:13:06:46 -0700] severity="INFORMATION" msgCount=17 msgID=org.opends.messages.backend-410 message="JE backend 'rA' does not specify the number of cleaner threads: defaulting to 8 threads"
[18/Oct/2024:13:06:46 -0700] severity="INFORMATION" msgCount=18 msgID=org.opends.messages.backend-411 message="JE backend 'rA' does not specify the number of lock tables: defaulting to 17"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=19 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_aci is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=20 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_entryUUID is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=21 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_objectClass is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=22 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_ds-sync-hist is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=23 msgID=org.opends.messages.backend-535 message="Due to changes in the configuration, index o=ra_ds-sync-conflict is currently operating in a degraded state and must be rebuilt before it can be used"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=24 msgID=org.opends.messages.backend-513 message="The database backend rA containing 0 entries has started"
[18/Oct/2024:13:06:46 -0700] severity="NOTICE" msgCount=25 msgID=org.opends.messages.backend-414 message="Import task 20241018130645873 finished execution in the state Stopped by error"
Import task 20241018130645873 did not complete successfully

Seems it makes no difference. Also, I don't think schema validation would account for these index errors.

@vharseko vharseko self-assigned this Oct 21, 2024
vharseko added a commit to vharseko/OpenDJ that referenced this issue Oct 22, 2024
… aborted because the entry does not have a parent entry"
@vharseko
Copy link
Member

@JesseCoretta
Copy link
Contributor Author

JesseCoretta commented Oct 23, 2024

@vharseko -- thank you. Will check once the builds are released, looks like its almost finished.

EDIT: build failed, though I see they're mostly MacOS (#445). Not seeing the release artifacts as a result. I'll check back in a few hours to see if the issue was resolved.

On a related note -- would this explain the build issues I experienced over the weekend? I tried to prepare #439 but unit tests were succeeding on my local system, and failing in our docker system.

vharseko added a commit to vharseko/OpenDJ that referenced this issue Oct 24, 2024
… aborted because the entry does not have a parent entry"
@vharseko
Copy link
Member

vharseko commented Oct 24, 2024

I tried to prepare #439 but unit tests were succeeding on my local system, and failing in our docker system.

- name: Set Integration Test Environment
id: failsafe
if: runner.os != 'Windows'
run: |
echo "MAVEN_PROFILE_FLAG=-P precommit" >> $GITHUB_OUTPUT
- name: Build with Maven
timeout-minutes: 180
env:
MAVEN_OPTS: -Dhttps.protocols=TLSv1.2 -Dmaven.wagon.httpconnectionManager.ttlSeconds=120 -Dmaven.wagon.http.retryHandler.requestSentEnabled=true -Dmaven.wagon.http.retryHandler.count=10
run: mvn --batch-mode --errors --update-snapshots verify --file pom.xml ${{ steps.failsafe.outputs.MAVEN_PROFILE_FLAG }}

use -Pprecommit switch

mvn clean install -Pprecommit -f ~/git/OpenIdentityPlatform/OpenDJ

@vharseko
Copy link
Member

EDIT: build failed, though I see they're mostly MacOS (#445). Not seeing the release artifacts as a result. I'll check back in a few hours to see if the issue was resolved.

fixed https://github.com/OpenIdentityPlatform/OpenDJ/actions/runs/11501698117

@JesseCoretta
Copy link
Contributor Author

@vharseko -- I cannot confirm. With that link, I only see a single downloadable build and it is MacOS.

@vharseko
Copy link
Member

MacOS-8 is cross platform

@JesseCoretta
Copy link
Contributor Author

@vharseko

Using the MacOS build produces some unusual results.

$ sudo ./setup --cli -b o=rA -D "cn=Registration Authority" -w "secret" -n --noPropertiesFile

Configuring Directory Server ..... Done.
Starting Directory Server ...... Done.

To see basic server configuration status and configuration, you can launch
/opt/opendj/bin/status
$ sudo /opt/opendj/bin/stop-ds
$ sudo /opt/opendj/bin/import-ldif -D "cn=Registration Authority" -w secret --backendID=userRoot --threadCount=1 --ldifFile ~/dev/tmp/radit.ldif --offline

[25/Oct/2024:05:23:19 -0700] category=JVM severity=NOTICE msgID=21 msg=Installation Directory:  /opt/opendj
[25/Oct/2024:05:23:19 -0700] category=JVM severity=NOTICE msgID=23 msg=Instance Directory:      /opt/opendj
[25/Oct/2024:05:23:19 -0700] category=JVM severity=NOTICE msgID=17 msg=JVM Information: 11.0.24+8-post-Ubuntu-1ubuntu322.04 by Ubuntu, 64-bit architecture, 4152360960 bytes heap size
[25/Oct/2024:05:23:19 -0700] category=JVM severity=NOTICE msgID=18 msg=JVM Host: hplptp, running Linux 6.8.0-47-generic amd64, 16607133696 bytes physical memory size, number of processors available 8
[25/Oct/2024:05:23:19 -0700] category=JVM severity=NOTICE msgID=19 msg=JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED", "-Dorg.opends.server.scriptName=import-ldif"
[25/Oct/2024:05:23:19 -0700] category=BACKEND severity=INFORMATION msgID=410 msg=JE backend 'userRoot' does not specify the number of cleaner threads: defaulting to 8 threads
[25/Oct/2024:05:23:19 -0700] category=BACKEND severity=INFORMATION msgID=411 msg=JE backend 'userRoot' does not specify the number of lock tables: defaulting to 17
[25/Oct/2024:05:23:19 -0700] category=BACKEND severity=NOTICE msgID=533 msg=OpenDJ Server 4.8.2-SNAPSHOT starting import (build 20241024175123, Rebb2ac6080c8281c28968e9a9bb8ef519aefb5d1)
[25/Oct/2024:05:23:19 -0700] category=BACKEND severity=NOTICE msgID=528 msg=The amount of free memory available to the import task is 3737124864 bytes. The number of phase one buffers required is 40 buffers
[25/Oct/2024:05:23:19 -0700] category=BACKEND severity=NOTICE msgID=520 msg=Setting DB cache size to 33554432 bytes and phase one buffer size to 28028436 bytes
[25/Oct/2024:05:23:20 -0700] category=BACKEND severity=NOTICE msgID=534 msg=Import Thread Count: 1 threads
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two started processing 1 buffers in 1 batches
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/entryUUID.uuidMatch phase two started processing 1 buffers in 1 batches
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/dn2id phase two started processing 1 buffers in 1 batches
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/objectClass.objectIdentifierMatch phase two processing completed
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/entryUUID.uuidMatch phase two processing completed
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/dn2id phase two processing completed
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=523 msg=Index /o=ra/id2childrencount phase two started processing 1 buffers in 1 batches
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=522 msg=Index /o=ra/id2childrencount phase two processing completed
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=531 msg=Total import time was 2 seconds. Phase one processing completed in 2 seconds, phase two processing completed in 0 seconds
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=519 msg=Processed 127610 entries, imported 3, skipped 0, rejected 127607 and migrated 0 in 3 seconds (average rate 41811.9/sec)
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=526 msg=Import LDIF environment close took 0 seconds
[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=518 msg=Flushing data to disk

Line of interest:

[25/Oct/2024:05:23:22 -0700] category=BACKEND severity=NOTICE msgID=519 msg=Processed 127610 entries, imported 3, skipped 0, rejected 127607 and migrated 0 in 3 seconds (average rate 41811.9/sec)

So it does not seem to be fixed. FWIW ... this is the same data you've seen previously.

@JesseCoretta
Copy link
Contributor Author

@vharseko -- ah, on a hunch, I tried your -S parameter:

[25/Oct/2024:05:42:41 -0700] category=BACKEND severity=NOTICE msgID=531 msg=Total import time was 8 seconds. Phase one processing completed in 7 seconds, phase two processing completed in 1 seconds
[25/Oct/2024:05:42:41 -0700] category=BACKEND severity=NOTICE msgID=519 msg=Processed 127609 entries, imported 127609, skipped 0, rejected 0 and migrated 0 in 10 seconds (average rate 11732.0/sec)
[25/Oct/2024:05:42:41 -0700] category=BACKEND severity=NOTICE msgID=526 msg=Import LDIF environment close took 0 seconds
[25/Oct/2024:05:42:41 -0700] category=BACKEND severity=NOTICE msgID=518 msg=Flushing data to disk

But what is curious is that the structure of this LDIF works in OpenLDAP, 389DS, et al ... so I am uncertain why -S would be needed.

@vharseko
Copy link
Member

I will check this case additionally

@vharseko
Copy link
Member

bin/import-ldif --backendID=rA --ldifFile ~/Downloads/content.ldif.txt --offline  -R /tmp/reject

cat /tmp/reject 
# Entry cn=itu-t,n=0,ou=Registrations,o=rA read from LDIF starting at line 23 is not valid because it violates the server's schema configuration: Entry cn=itu-t,n=0,ou=Registrations,o=rA violates the Directory Server schema configuration because it is missing attribute n which is required by objectclass registration
objectClass: top
objectClass: extensibleObject
objectClass: subentry
objectClass: collectiveAttributeSubentry
objectClass: spatialContext
cn: itu-t
subtreeSpecification: {minimum 1}
c-topArc;collective: n=0,ou=Registrations,o=rA

use --skipSchemaValidation or switch to online import

@JesseCoretta
Copy link
Contributor Author

Fine

@vharseko vharseko changed the title The import has been aborted because the entry 'n=0,ou=Registrations,o=rA' does not have a parent entry The import has been aborted because the entry does not have a parent entry Oct 29, 2024
vharseko added a commit that referenced this issue Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants