Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Janusupgrade to 1.0.0 with a smart marker logic to reduce Cassandra Calls #3849

Open
wants to merge 55 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
364c4aa
PLT-2568 optimise vertex fetch from janusgraph
sriram-atlan Nov 13, 2024
8c7ca54
remove method
aarshi0301 Nov 19, 2024
3df42f5
Fix property name
aarshi0301 Nov 18, 2024
0bb62fc
add null cehck
aarshi0301 Nov 18, 2024
9b966a6
remove submit call
aarshi0301 Nov 18, 2024
b0530b3
handle cocurrent modification exception
aarshi0301 Nov 18, 2024
52e645d
update libraries
aarshi0301 Nov 18, 2024
c323ad7
remove scrub results
aarshi0301 Nov 18, 2024
9741f59
fix results
aarshi0301 Nov 19, 2024
1545164
add dummy commit
aarshi0301 Nov 19, 2024
8259f67
Merge pull request #3767 from atlanhq/dummy
aarshi0301 Nov 19, 2024
953bce5
add branch
aarshi0301 Nov 19, 2024
c6c284b
add janus branch
aarshi0301 Nov 19, 2024
d27ea4d
scrub results
aarshi0301 Nov 19, 2024
0015ef7
add flag for janusgraph optimisation
aarshi0301 Nov 19, 2024
54103c8
add flag while retrieving from cassandra
aarshi0301 Nov 19, 2024
b903c7b
log ES query error
aarshi0301 Nov 19, 2024
dc77657
fix map vertex method
aarshi0301 Nov 20, 2024
ec4a32d
add configuration in flag
aarshi0301 Nov 20, 2024
f51f8f6
fix return type
aarshi0301 Nov 20, 2024
391e510
add flag for collapsed result
aarshi0301 Nov 20, 2024
8751387
fix updated by and createdby field
aarshi0301 Nov 20, 2024
6f1549d
fix if condition
aarshi0301 Nov 21, 2024
7feef65
remove branch deployment
aarshi0301 Nov 21, 2024
bd1ddb0
use multi query to improve response time of post elastic query lookups
sriram-atlan Nov 21, 2024
0bc01d6
Merge remote-tracking branch 'origin/janusoptimisation' into janusopt…
sriram-atlan Nov 21, 2024
650cfd1
import function
aarshi0301 Nov 21, 2024
f20e859
remove the mapVertex method that uses valueMap
sriram-atlan Nov 21, 2024
0022dae
build this branch as docker image
sriram-atlan Nov 21, 2024
1be3658
close thread bound janusgraph transaction
sriram-atlan Nov 21, 2024
d73d858
revert entity audit listener
sriram-atlan Nov 21, 2024
0af18e1
multi query with janusgraph 1.0.0
sriram-atlan Nov 22, 2024
4e39d43
add attributes part of multi query
sriram-atlan Nov 23, 2024
9c5a643
include janusoptimisation-multiQuery-1.0.0 in ecr build push
sriram-atlan Nov 23, 2024
01e5b3b
include janusoptimisation-1.0.0 in ecr build push
sriram-atlan Nov 23, 2024
ee86e6e
include janusupgrade in ecr build push
sriram-atlan Nov 23, 2024
e042b4f
bulk fetch with auto batching
sriram-atlan Nov 25, 2024
9ee5645
use properties method with janusgraph than gremlin
sriram-atlan Dec 7, 2024
c8578e4
use edge fetch always
sriram-atlan Dec 7, 2024
a47bdee
log edge and edge properties
sriram-atlan Dec 7, 2024
4dd2743
lookup only when marker exists or when attribute relationship is inwards
sriram-atlan Dec 7, 2024
76fe1c8
include all policy attributes in prefetch exclude
sriram-atlan Dec 7, 2024
358a7bc
revert using prefetch in mapSystemAttributes and toAtlasToObjectId
sriram-atlan Dec 7, 2024
5c6ad1b
remove logs. update comments
sriram-atlan Dec 7, 2024
b95c0e6
fix business attribute
aarshi0301 Dec 9, 2024
4cb985a
add check for business attributes
aarshi0301 Dec 10, 2024
8eef377
code cleanup
aarshi0301 Dec 11, 2024
3b009cd
add upgrade changes
aarshi0301 Dec 11, 2024
d97603d
fix build
aarshi0301 Dec 11, 2024
a6a7b92
fix build
aarshi0301 Dec 11, 2024
2692ae2
fetch only active edges
aarshi0301 Dec 12, 2024
b4caf2b
Minor refactoring
aarshi0301 Dec 13, 2024
9de9e60
replace in and out with both
aarshi0301 Dec 13, 2024
aa040f6
remove guid
aarshi0301 Dec 13, 2024
286943e
Merge branch 'master' into janusupgrade
aarshi0301 Jan 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add flag for janusgraph optimisation
aarshi0301 committed Nov 19, 2024
commit 0015ef7762eb9ea519b921584b144839d306ad42
9 changes: 9 additions & 0 deletions intg/src/main/java/org/apache/atlas/AtlasConfiguration.java
Original file line number Diff line number Diff line change
@@ -113,7 +113,16 @@ public enum AtlasConfiguration {
HERACLES_API_SERVER_URL("atlas.heracles.api.service.url", "http://heracles-service.heracles.svc.cluster.local"),

INDEXSEARCH_ASYNC_SEARCH_KEEP_ALIVE_TIME_IN_SECONDS("atlas.indexsearch.async.search.keep.alive.time.in.seconds", 300),

/**
* hits elastic search async APU
*/
ENABLE_ASYNC_INDEXSEARCH("atlas.indexsearch.async.enable", false),

/***
* enables parallel processing of janus graph vertices from cassandra
*/
ENABLE_JANUS_GRAPH_OPTIMISATION("atlas.janus.graph.optimisation.enable", false),
ATLAS_INDEXSEARCH_QUERY_SIZE_MAX_LIMIT("atlas.indexsearch.query.size.max.limit", 100000),
ATLAS_INDEXSEARCH_LIMIT_UTM_TAGS("atlas.indexsearch.limit.ignore.utm.tags", ""),
ATLAS_INDEXSEARCH_ENABLE_API_LIMIT("atlas.indexsearch.enable.api.limit", false),