All New WineDirect Data Lake

The All New WineDirect Data Lake provides all Order, Customer and Wine Club records from All New WineDirect.

Data Refresh Frequency

All New WineDirect Order, Customer, and Wine Club data are updated hourly. Folders for each day will contain orders that were CREATED OR MODIFIED on that date. An order can appear in multiple date folders if it gets updated on multiple dates. All dates and times are in UTC. All New WineDirect hourly data populates 5-10 minutes after the end of the hour.

Orders 

Data Structure 

Order Schema 

Files are in Parquet format.

Sample orders parquet file with 50 dummy orders record

Orders Sample File

Folder Structure 

Data is partitioned by Tenant ID, Year, Month, Day. Files can be downloaded for a specified year, month, or day.

URL Structure 

For new or migrated websites on ANWD, use the Tenant ID to access orders and all other objects (customers, products, etc.) listed below.

https://datalake.alpha.app.winedirect.com/tenant/{tenant_id}/orders/{year}/{month}/{day}

IMPORTANT NOTE: To access client data on the ANWD staging environment, use the Tenant ID and staging URL to access orders and all other objects (customers, products, etc.) listed below.

Hourly partitioned data is available in ANWD. Pass the hour to the URL to fetch the hourly data. A 5-10 minute lag exists before the previous hour's data is populated. 

- Staging URL:
https://datalake.alpha.staging.app.winedirect.com/tenant/{tenant_id}/orders/{year}/{month}/{day}/{hour} 

- Production URL:
https://datalake.alpha.app.winedirect.com/tenant/{tenant_id}/orders/{year}/{month}/{day}/{hour} 

Examples 

• For Tenant ID abcd-123-XYZ-345, to get all orders from January 2023 use this URL (All small alphabets for tenant ID) 

  https://datalake.alpha.app.winedirect.com/tenant/abcd-123-XYZ-345/orders/2023/1  

• For Tenant ID abcd-123-XYZ-345, to get all orders from 2022 use this URL (All small alphabets for tenant ID)
  https://datalake.alpha.app.winedirect.com/tenant/abcd-123-xyz-345/orders/2022 
• For Tenant ID abcd-123-XYZ-345, to get all orders from 2023-11-09 hour 15 to 16 UTC use this URL (All small alphabets for tenant ID)
  https://datalake.alpha.app.winedirect.com/tenant/abcd-123-xyz-345/orders/2023/11/9/15
• For Tenant ID abcd-123-XYZ-345, to get all orders from 15th January 2020 use this URL  https://datalake.alpha.app.winedirect.com/tenant/abcd-123-XYZ-345/orders/2020/1/15
  

Data Refresh Frequency  

Data is updated daily. Folders for each day will contain orders that were CREATED OR MODIFIED on that date. An order can appear in multiple date folders if it gets updated on multiple dates. All dates and times are in UTC. ANWD Orders Hourly data populates 5-10 min after the end of the hour.

Steps to Download Records from Data Lake  

 1. Each Tenant will have its own credentials. If you have an ANWD store, a user with StoreAdmin access can get the credentials from the Integrations section on the Admin Panel

2. Authenticate using the credentials <client_id> and <client_secret> provided

Text
curl -X POST \
--user <client_id>:<client_secret> \
'https://datalake.alpha.app.winedirect.com/oauth2/token?grant_type=client_credentials' \
-H 'Content-Type: application/x-www-form-urlencoded'

3.  Authentication will return a token to use for accessing data. Use the token to fetch data from the Data Lake. The token will be valid for 60 min. 

Text
curl -X GET \
-H "Authorization: Bearer <TOKEN>" \
-H "Content-Type: application/json" https://datalake.alpha.app.winedirect.com/tenant/{tenant_id}/orders/{year}/{month}/{day} \
--output results.zip

Customers

The Authentication process for Customer data is the same as the process for Orders data. The same token can be used if it is still valid. 

Below are the URLs available to download Customer related data:

Customers

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/customers/<yyyy>/<mm>/<dd>
Customers Data Lake Schema 

Sample customers parquet file with 50 dummy customer records - Sample Customer Parquet File

Customer Tags

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/customer_tags/<yyyy>/<mm>/<dd>
Customers Tags Data Lake Schema 

Customer Segments

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/customer_segments/<yyyy>/<mm>/<dd>
Customer Segments Data Lake Schema 

Customer Date Type Options

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/customer_date_type_options/<yyyy>/<mm>/<dd>
Customer Date Type Options Schema 

Customer Groups

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/customer_groups/<yyyy>/<mm>/<dd>
Customers Groups Data Lake Schema 

Wine Clubs

The authentication process for Wine Clubs data is the same as for Orders; the same token can be used if it is still valid. 

Below are the URLs available to download Wine Clubs related data:

Clubs

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/clubs/<yyyy>/<mm>/<dd>
Clubs Data Lake Schema 

Sample Clubs parquet file with 50 dummy clubs records - Sample Clubs Parquet File

Clubs Memberships

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/clubs_memberships/<yyyy>/<mm>/<dd>
Clubs Memberships Data Lake Schema 

Clubs Releases Products

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/clubs_release_products/<yyyy>/<mm>/<dd>
Clubs Releases Products Data Lake Schema 

Clubs Releases

https://datalake.alpha.app.winedirect.com/tenant/<tenantid>/clubs_releases/<yyyy>/<mm>/<dd>
Clubs Releases Data Lake Schema