Skip to content

Commit

Permalink
Add IPAddress Presto type (facebookincubator#10596)
Browse files Browse the repository at this point in the history
Summary:
Split of facebookincubator#10538

In response to mbasmanova :
facebookincubator#10538 (review)

Took a look and added the to and from varbinary casting
Contains CAST functions for IPAddress <-> Varchar and IPAddress <-> Varbinary utilizing the folly library.

In response to:
https://velox-lib.io/blog/optimize-try-more

I believe the folly library utilizes makeUnexpected already.
https://github.com/facebook/folly/blob/main/folly/IPAddress.cpp
I added one error for Varbinary conversion utilizing `context.setStatus`
If I'm not understanding something correctly please let me know.

As for adding a limited fuzzer aditi-pandit
facebookincubator#10538 (comment)

Is this to test the casting `IPAddress <-> Varchar and IPAddress <-> Varbinary` or something else.
We already test the casting using the duckdb type, we also have the prestissimo test in the E2E test.
Fuzzer support will be added in after the types and functions have been added.

Pull Request resolved: facebookincubator#10596

Reviewed By: bikramSingh91

Differential Revision: D61163377

Pulled By: xiaoxmeng

fbshipit-source-id: 2c0fa2e8e6d250a2167a1ff82b2878548798a226
  • Loading branch information
mohsaka authored and facebook-github-bot committed Aug 13, 2024
1 parent 2abfadf commit e092535
Show file tree
Hide file tree
Showing 14 changed files with 729 additions and 4 deletions.
9 changes: 9 additions & 0 deletions velox/docs/develop/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ HYPERLOGLOG VARBINARY
JSON VARCHAR
TIMESTAMP WITH TIME ZONE BIGINT
UUID HUGEINT
IPADDRESS HUGEINT
======================== =====================

TIMESTAMP WITH TIME ZONE represents a time point in milliseconds precision
Expand All @@ -146,6 +147,14 @@ Supported range of milliseconds is [0xFFF8000000000000L, 0x7FFFFFFFFFFFF]
store timezone ID. Supported range of timezone ID is [1, 1680].
The definition of timezone IDs can be found in ``TimeZoneDatabase.cpp``.

IPADDRESS represents an IPV6 or IPV4 formatted IPV6 address. Its physical
type is HUGEINT. The format that the address is stored in is defined as part of `(RFC 4291#section-2.5.5.2) <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.5.5.2>`_
As Velox is run on Little Endian systems and the standard is network byte(Big Endian)
order, we reverse the bytes to allow for masking and other bit operations
used in IPADDRESS/IPPREFIX related functions. This type can be used to
create IPPREFIX networks as well as to check IPADDRESS validity within
IPPREFIX networks.

Spark Types
~~~~~~~~~~~~
The `data types <https://spark.apache.org/docs/latest/sql-ref-datatypes.html>`_ in Spark have some semantic differences compared to those in
Expand Down
Loading

0 comments on commit e092535

Please sign in to comment.