Feast v0.10.7

Fixed bugs:

• Fix race condition in historical e2e tests #1620 (woop)

Merged pull requests:

• Use drop_duplicates() instead of groupby (about 1.5~2x faster) #1617 (rightx2)
• Use CONCAT() instead of ROW_NUMBER() #1601 (MattDelac)
• Minor doc fix in the code snippet: Fix to reference the right instance for the retrieved job instance object #1599 (dmatrix)
• Append nanoseconds to dataset name in test_historical_retrival to prevent tests stomping over each other #1593 (achals)
• Make start and end timestamps tz aware in the CLI #1590 (achals)

Feast v0.10.6

Implemented enhancements:

• Add datastore namespace option in configs #1581 (tsotnet)

Fixed bugs:

• Fix contention issue #1582 (woop)
• Ensure that only None types fail predicate #1580 (woop)
• Don't create bigquery dataset if it already exists #1569 (tsotnet)

Merged pull requests:

• Change OfflineStore class description #1571 (tedhtchang)

Feast v0.10.5

Implemented enhancements:

• Add offline_store config #1552 (tsotnet)

Fixed bugs:

• Validate project and repo names for apply and init commands #1558 (tedhtchang)

Merged pull requests:

• Fix Sphinx documentation building #1563 (woop)
• Add test coverage and remove MacOS integration tests #1562 (woop)
• Improve GCP exception handling #1561 (woop)
• Update default cli no option help message #1550 (tedhtchang)
• Add opt-out exception logging telemetry #1535 (jklegar)
• Add instruction for install Feast on IKS and OpenShift using Kustomize #1534 (tedhtchang)

Feast v0.10.4

Implemented enhancements:

• Inferencing of Features in FeatureView and timestamp column of DataSource #1523 (mavysavydav)
• Add Unix Timestamp value type #1520 (MattDelac)
• Fix materialize for None #1481 (qooba)

Merged pull requests:

• BigQuery type to Feast type conversion chart update #1530 (mavysavydav)
• remove unnecessay path join in setup.py #1529 (shihabuddinbuet)
• Add roadmap to documentation #1528 (woop)
• Add test matrix for different Python versions #1526 (woop)
• Update broken urls in the github pr template file #1521 (tedhtchang)
• Upgrade grpcio version in python SDK #1508 (szalai1)
• Better logging for materialize command #1499 (jklegar)

Feast v0.10.3

Implemented enhancements:

• Add support for third party providers #1501 (tsotnet)
• Infer entity dataframe event timestamp column #1495 (jklegar)
• Allow Feast apply to import files recursively (and add .feastignore) #1482 (tsotnet)

Fixed bugs:

• Bump supported Python version to 3.7 #1504 (tsotnet)
• Fix bug in allowing empty repositories to be applied to a GCS registry #1488 (woop)

Merged pull requests:

• Add a fixed timestamp to quickstart data #1513 (jklegar)
• Make gcp imports optional #1512 (jklegar)
• Fix documentation inconsistency #1510 (jongillham)
• pre-commit command typo fix in CONTRIBUTING.md #1506 (mavysavydav)
• Add optional telemetry to other CLI commands #1505 (jklegar)
• Pass entities information to Provider #1498 (MattDelac)
• Update broken urls in contributing.md #1489 (tedhtchang)
• Python docs formatting fixes #1473 (jklegar)

Feast v0.10.2

Fixed bugs:

• Fix bug in allowing empty repositories to be applied to a GCS registry #1488 (woop)

Feast v0.10.1

Fixed bugs:

• Fix time zone issue with get_historical_features #1475 (tsotnet)

Merged pull requests:

• Improve exception handling, logging, and validation #1477 (woop)
• Remove duped pic #1476 (YikSanChan)
• Fix created timestamp related errors for BigQuery source #1474 (jklegar)
• Remove unnecessary MAVEN_CONFIG #1472 (danielsiwiec)
• Fix CLI entities command & add feature-views command #1471 (tsotnet)

Feast v0.10.0

Overview

Feast 0.10 brings major improvements towards making Feast an easy to use and operate feature store. Feast 0.10 ships with a lightweight SDK/CLI that allows teams to stand up a local or production feature store within seconds. This latest release also comes with extensibility in its provider model, allowing teams to easily add new data stores, compute layers, or cloud providers.

👥 Contributors

Thanks to @jklegar, @oavdeev, @tsotnet, @woop, @terryyylim, @randxie, @josegpg, @mrzzy, @pyalex, @tedhtchang, @theofpa, @YikSanChan and @jparthasarthy for the pull requests and contributions!

✨ New Features:

Add template generation to Feast CLI for Google Cloud Platform #1460 (woop)
Add support for retrieving data from sources that don't match providers #1454 (woop)
Add materialize-incremental CLI command #1442 (tsotnet)
Add registry refreshing and caching #1431 (woop)
Add missing FeatureStore methods #1423 (jklegar)
Allow importing of new FeatureStore classes #1422 (woop)
Add Feast init command #1414 (oavdeev)
Add support for parquet ingestion #1410 (oavdeev)
Add materialize_incremental method #1407 (jklegar)
Add support for pull query from BigQuery #1403 (jklegar)
Add support for partial apply to create infra #1402 (oavdeev)
Add online read API to FeatureStore class #1399 (oavdeev)
Add historical retrieval for BigQuery and Parquet #1389 (woop)
Add feature views #1386 (oavdeev)
Implement materialize method #1379 (jklegar)
Read and write path for Datastore and SQLite #1376 (oavdeev)
Download BigQuery table to pyarrow table for python-based ingestion flow #1366 (jklegar)
FeatureStore, FeatureView, Config, and BigQuerySource classes for updated SDK #1364 (jklegar)
Add support for new deploy CLI #1362 (oavdeev)

🔴 Fixed bugs:

Fix time zone access with native python datetimes #1469 (tsotnet)
Small fixes for created_timestamp #1468 (jklegar)
Fix offline store (tz-naive & field_mapping issues) #1466 (tsotnet)
Fix get_online_features return schema #1455 (jklegar)
Fix noisy path warning #1452 (woop)
Fix flaky test_feature_store fixture #1447 (jklegar)
Use timestamp check for token refresh #1444 (terryyylim)
Fix bug in event timestamp removal in local mode #1441 (jklegar)
Fix timezone issue in materialize & materialize_incremental #1439 (tsotnet)
Fix materialization_intervals initialization in FeatureView #1438 (tsotnet)
Fix broken Terraform installation files #1420 (josegpg)
Fix retry handling for GCP datastore #1416 (oavdeev)
Make CLI apply in local mode idempotent #1401 (oavdeev)
Fix a bug in client archive_project method and fix lint in grpc auth #1396 (randxie)

🔨 Merged pull requests:

Change GCP template names to match local template #1470 (jklegar)
Add logging to materialize #1467 (woop)
Validate timestamp column present in entity dataframe #1464 (jklegar)
Fix & clean up Feast CLI commands #1463 (tsotnet)
Flatten configuration structure for online store #1459 (woop)
Optimize write rate in Gcp Firestore #1458 (tsotnet)
Allow apply to take a single Entity or FeatureView #1457 (jklegar)
Validate datetimes in materialize in correct order #1456 (jklegar)
Add test to ensure saving and loading from registry is safe #1453 (woop)
Port telemetry to FeatureStore API #1446 (jklegar)
Add materialize-incremental cli test #1445 (tsotnet)
Support join keys in historical feature retrieval #1440 (jklegar)
Refactor OfflineStore into Provider #1437 (woop)
Fix multi-entity online retrieval #1435 (woop)
Fix feature name consistency between online & historical apis #1434 (tsotnet)
Rename Metadata Store to Registry #1433 (woop)
Add support for Pydantic as configuration loader #1432 (woop)
Add entity join key and fix entity references #1429 (jklegar)
Slightly more sensible test names #1428 (oavdeev)
Make entity description optional and fix empty table_ref #1425 (jklegar)
Add Development Guide for Main Feast Repo Feast Components #1424 (mrzzy)
Fix protobuf building for Python SDK #1418 (woop)
Add project name generator #1417 (woop)
[Python SDK][Auth] Refresh token id w/o gcloud cli #1413 (pyalex)
Firestore ingestion perf improvements + benchmark script #1411 (oavdeev)
Fixed old urls in documentation #1406 (tedhtchang)
Upgrade Gcloud setup dependency #1405 (woop)
Fix documentation building for Feast SDK #1400 (woop)
Bump jinja2 from 2.11.2 to 2.11.3 in /sdk/python #1398 (dependabot[bot])
Improve spark-on-k8s-operator documentation #1397 (jklegar)
Update Python SDK dependencies #1394 (woop)
Move Python proto package into submodule #1393 (woop)
Add nicer validation for repo config #1392 (oavdeev)
Remove Python CI dependencies #1390 (woop)
Move Project field to Table/View spec #1388 (woop)
Remove Mirror CI #1387 (woop)
Add feedback link to install docs page #1375 (jparthasarthy)
Support multiple features per key in firestore format spec #1374 (oavdeev)
Fix hashing algorithm in the firestore spec #1373 (oavdeev)
Support make protos on Mac #1371 (tedhtchang)
Add support for privileged tests #1369 (woop)
Remove base tests folder #1368 (woop)
Add Firestore online format specification #1367 (oavdeev)
Improve documentation for k8s-spark resource template #1363 (theofpa)

Feast v0.9.1

Changelog

v0.9.1 (2021-01-29)

Full Changelog

Implemented enhancements:

• Add telemetry to Python SDK #1289 (jklegar)

Fixed bugs:

• Fix kafka download url #1298 (jklegar)
• disable telemetry in docker-compose test and job_service #1297 (jklegar)

Feast v0.9.0

Full Changelog

Implemented enhancements:

• Enable user to provide spark job template as input for jobservice deployment #1285 (khorshuheng)
• Add feature table name filter to jobs list api #1282 (terryyylim)
• Report observed value for aggregated checks in pre-ingestion feature validation #1278 (pyalex)
• Add docs page for Azure setup #1276 (jklegar)
• Azure example terraform #1274 (jklegar)

Fixed bugs:

• make EMR jar uploader work the same as k8s one #1284 (oavdeev)
• Don't error when azure vars not set #1277 (jklegar)
• Prevent ingestion job config parser from unwanted fieldMapping transformation #1261 (pyalex)
• Features are not being ingested due to max age overflow #1209 (pyalex)
• Feature Table is not being update when only max_age was changed #1208 (pyalex)
• Truncate staging timestamps in entities dataset to ms #1207 (pyalex)
• Bump terraform rds module version #1204 (oavdeev)

Merged pull requests:

• Use date partitioning column in FileSource #1293 (pyalex)
• Add EMR CI/CD entrypoint script #1290 (oavdeev)
• Online serving optimizations #1286 (pyalex)
• Make third party grpc packages recognizable as python module #1283 (khorshuheng)
• Report observed values in feature validation as Gauge #1280 (pyalex)
• Keep same amount of partitions after repartitioning in IngestionJob #1279 (pyalex)
• Add request feature counter metric #1272 (terryyylim)
• Use SEND_INTERRUPT to cancel EMR jobs #1271 (oavdeev)
• Fix historical test flakiness #1270 (jklegar)
• Allow https url for spark ingestion jar #1266 (jklegar)
• Add project name to feature validation metric #1264 (pyalex)
• Use dataproc console url instead of gcs for log uri #1263 (khorshuheng)
• Make nodes priority (for redis cluster) configurable in Serving #1260 (pyalex)
• Enhance job api to return associated feature table and start time #1259 (khorshuheng)
• Reporting metrics from validation UDF #1256 (pyalex)
• Allow use the same timestamp column for both created & even timestamp in Historical Retrieval #1255 (pyalex)
• Apply grpc tracing interceptor on Feast SDK #1243 (khorshuheng)
• Apply grpc tracing interceptor on online serving #1242 (khorshuheng)
• Python UDF in Ingestion being used for feature validation #1234 (pyalex)
• Add spark k8s operator launcher #1225 (oavdeev)
• Add deadletter/read-from-source metrics to batch and stream ingestion #1223 (terryyylim)
• Implement AbstractStagingClient for azure blob storage #1218 (jklegar)
• Configurable materialization destination for view in BigQuerySource #1201 (pyalex)
• Update Feast Core list features method #1176 (terryyylim)
• S3 endpoint configuration #1169 #1172 (mike0sv)
• Increase kafka consumer waiting time in e2e tests #1268 (pyalex)
• E2E tests support for jobservice's control loop #1267 (pyalex)
• Optimize memory footprint for Spark Ingestion Job #1265 (pyalex)
• Fix historical test for azure #1262 (jklegar)
• Change azure https to wasbs and add azure creds to spark #1258 (jklegar)
• Docs, fixes and scripts to run e2e tests in minikube #1254 (oavdeev)
• Fix azure blob storage access in e2e tests #1253 (jklegar)
• Update python version requirements to 3.7 for Dataproc launcher #1251 (pyalex)
• Fix build-ingestion-py-dependencies script #1250 (pyalex)
• Add datadog(statsd) client to python package for IngestionJob #1249 (pyalex)
• Add prow job for azure e2e test #1244 (jklegar)
• Azure e2e test #1241 (jklegar)
• Add Feast Serving histogram metrics #1240 (terryyylim)
• CI should work on python 3.6 #1237 (pyalex)
• Integration test for k8s spark operator support #1236 (oavdeev)
• Add prow config for spark k8s operator integration testing #1235 (oavdeev-tt)
• Upgrading spark to 3.0.1 #1227 (pyalex)
• Support TFRecord as one of the output formats for historical feature retrieval #1222 (khorshuheng)
• Remove stage_dataframe from the launcher interface #1220 (oavdeev)
• Refactor staging client uploader and use it in EMR launcher #1219 (oavdeev)
• Remove unused EMR code #1217 (oavdeev)
• Remove job id from ingested row counter metric #1216 (terryyylim)
• Quickstart link fixed #1213 (szczeles)
• Delete v1 concepts #1194 (terryyylim)
• Dont write defaults to config #1188 (mike0sv)
• Refactor tests which utilizes feature sets #1186 (terryyylim)
• Refactor configurable options and add sphinx docs #1174 (terryyylim)
• Remove unnecessary Google Auth dependency #1170 (woop)