Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Private link tutorial #490

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
69448b1
draft private link work
fiquick Oct 11, 2024
3df067e
edits
fiquick Oct 11, 2024
b27464d
edit
fiquick Oct 11, 2024
d54e795
change "customer" to "your"
fiquick Oct 11, 2024
aee36e4
text edits
fiquick Oct 14, 2024
88c823b
typo
fiquick Oct 14, 2024
7b60b96
typo
fiquick Oct 14, 2024
6336bed
typo
fiquick Oct 14, 2024
9348a1f
edit from feedback
fiquick Oct 16, 2024
d4acd36
private service to private link
fiquick Oct 16, 2024
265588d
private link
fiquick Oct 16, 2024
41430a9
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
628dbd9
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
f50a8b6
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
b3d875f
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
4a567f0
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
5959e93
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
8b6def1
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 17, 2024
3159d34
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 25, 2024
a939eb2
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 25, 2024
c647502
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 25, 2024
5aae411
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 25, 2024
f4a815c
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 25, 2024
887f3a1
Update modules/ROOT/pages/tutorials/private-link.adoc
fiquick Oct 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions modules/ROOT/content-nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,4 @@ AuraDS End
// * xref:tutorials/performance-improvements.adoc[]
* xref:tutorials/troubleshooting.adoc[]
* xref:tutorials/create-auradb-instance-from-terminal.adoc[]
* xref:tutorials/private-link.adoc[]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
122 changes: 122 additions & 0 deletions modules/ROOT/pages/tutorials/private-link.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
[[aura-private-link-tutorial]]
= Private link tutorial

label:AuraDB-Virtual-Dedicated-Cloud[]

This section is cloud-agnostic and therefore applicable to all clouds.

The term “private link” refers to:

* Private Service Connect = Google Cloud platform
* PrivateLink = AWS
* Private Link = Azure

This tutorial walks through the process of setting up a private link connection to securely connect your application to an AuraDB Virtual Dedicated Cloud environment.
A private link provides secure network connectivity between your application and AuraDB without exposing traffic to the public internet.
Refer to Figure 1 for a visual representation of the network package traffic flow.

.The network package traffic flow when an application connects to an Aura instance.
image::aura-private-link-architecture.png[Fig.1 Aura Private Link/Private Service Connect Architecture]

== Steps
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These steps are not steps that the user can take and maybe not necessary in a tutorial. They could move to another section where we would talk about the internals of private links.


[NOTE]
======
Thw dbid: abcd1234 and orch-id: 0000 are used in this tutorial.
These are different in your AuraDB Virtual Dedicated Cloud environment.
======

. The application initializes a driver connection to neo4j+s://abcd1234.production-orch-0000.neo4j.io.
. The network layer then queries the DNS server to resolve the fully qualified domain name (FQDN) (in this case, abcd1234.production-orch-0000.neo4j.io) to its corresponding IP address.
. The Cloud Virtual Network private DNS is queried, and it resolves the FQDN to 10.10.10.10, based on the wildcard DNS A record created: *.production-orch-0000.neo4j.io -> 10.10.10.10
. The application's connection is directed to 10.10.10.10, which is the private link endpoint.
From there, the private link endpoint forwards the network connection to the private ingress through the private link.
. The private ingress extracts the dbid from the FQDN and directs the connection to the appropriate Aura instance (dbid: abcd1234).
. The Aura instance responds by sending the Neo4j cluster routing table back to the application, which contains information about the available nodes and their roles (see Table 1 below).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use the word node in this context.

. The application then selects an appropriate node to execute a read or write transaction. In the case of a write transaction, the application sends the transaction to abcd1234-core-1.production-orch-0000.neo4j.io.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above. Also wondering why the dbid contains the word core.


Based on the type of transaction (read or write) the driver selects an appropriate node to execute a read or write transaction.
The code has the ability to direct the transaction to the appropriate nodes this way.

. Similar to before, the Cloud Virtual Network private DNS is queried and resolves the FQDN to 10.10.10.10. The application's connection is sent to the private link endpoint (10.10.10.10), which forwards the network connection to the private ingress through the private link.
The private ingress then directs the connection to the Aura instance with dbid: abcd1234.
. Finally, the write transaction is received and executed within the Aura instance with dbid: abcd1234.

.Available nodes and their roles
[cols="1,1"]
|===
|abcd1234-core-1.production-orch-0000.neo4j.io
|role: write

|abcd1234-core-2.production-orch-0000.neo4j.io
|role: read

|abcd1234-core-3.production-orch-0000.neo4j.io
|role: read
|===

== Summary

When you create a private link using the Aura console, the private ingress becomes accessible to all projects within the cloud provider’s VPC.

*The private link maps to an Orchestra, not a project.
You could have multiple orchestras (usually multiple regions)*

For example, if you’re using AWS, you can identify this private ingress by its unique name in your AWS environment.

First, create a private ingress service in Aura VPC, then create a cloud endpoint and try to connect to the service provided, then this private link will be accepted.
This can be completed on the Aura console.

And then there’s a trickier step, but it's the most important thing.
*The private DNS.*
You should configure this private DNS.
If you are using AWS, then AWS will create this private DNS for you.
For Google or Azure, you have to manually create this private DNS.
Add the Wildcard DNS entry *.production-orch-0000.neo4j.io and point it to the cloud IP endpoint.
Comment on lines +70 to +75
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a rewrite.



== Cloud endpoints

A cloud endpoint is essentially a network forwarding rule.
It receives network packets, but doesn't open them and forwards them to the Neo4j private ingress via the private link.
However, the private ingress only accepts packets from a recognized FQDN.

For example, http://abcd1234-core-1.production-orch-0000.neo4j.io/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again with the core.


There might be some other instances running in the same customer Aura environment.
We use this Access Point QDN (Qualified Domain Name) to figure out which Aura instance it is trying to reach.
[IMPORTANT]
====
Only FQDN is accepted and **not** the IP address.
====

== Private DNS

Once the private ingress, private link, and private endpoint with the DNS entry are configured, the application will attempt to reach the IP address.

And then it will try to go and check the private link, like “hey, what is the private DNS address for this AFQDN?”
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a work in progress, I assume?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, maybe it can say:

"Once the private ingress, private link, and private endpoint with the DNS entry are configured, the application will attempt to reach the IP address. Then it will check the private link, to check the private DNS address for the AFQDN.

Then the private DNS will tell the application to go to the IP address of the endpoint."

Thanks for all the feedback @AlexicaWright 🙌

Then the private DNS will tell the application, “you should go to 10.10.10 here.
This is the IP address of the endpoint."
Then this endpoint will just forward everything to Aura.

*Important: The private DNS only works for this single cloud VPC.*
Outside of this VPC that private DNS entry does not work, _that’s the reason why it’s called private DNS._

If your application is running in a different VPC, it will not be able to reach the endpoint.
This is because there is no DNS resolution across VPCs.
Setting up the private link within the same VPC is straightforward and typically succeeds without issues. However, configuring DNS for access across different VPCs or environments is more complex and requires careful setup.

If you want your application running on your own on-prem data centers you still can use
private link.
But you don’t want to go to the internet and then back to your own Aura instance because it's unsafe (data leakage, unauthorized access, etc.) and the latency might be too long.
If you still want applications running on your on-prem data center can reach out through this private link.

To enable communication from on-prem data centers, the first step is to create a cloud VPN or another type of secure connection between your on-prem network and the VPC that has the private link endpoint running in it.
You also need to configure your DNS server so that traffic directed to the FQDN is forwarded to the endpoint’s IP address.

== Other info

* Database upload is not possible whilst the public traffic is disabled.
* Bloom - We display it clearly outside the VPC, and we should explain that to access it the VPN needs to be established in "split-mode"
* [BONUS] LOAD CSV - Can we show / display the path used ? this will help customers understand how and where to place their data
* [BONUS] SSO - I'd love us to show what route is necessary to reach the IdP