Updated metrics tables/reports have arrived on October 18, 2021! Some reports may take 24-48 hours to reflect the new data after deploy due to sync timing.

Data dictionary has been updated to reflect the latest updates as well.

Updated Tables - For multi-tenant

1. Events - Dataset or Project Views By Org - Name changed (from “Events - Views by Org”) and column name “dataset_views” changed to “views”
2. Events - Searches - Last 90 Days - Fixed a bug that sometimes caused duplicate rows
3. Membership - Daily Counts - By Org - Name changed (from “Membership - Daily - By Org")
4. Resources - Org Owned Database connections - Name changed (from “Resources - Database connections”) and added column “owner”
5. Tops - Bookmarks - Extended range to all users (it previously was limited to the top 10 users) and added column “displayname”
6. Tops - Dataset Creation - Extended range to all users (it previously was limited to the top 10 users) and added column “displayname”
7. Tops - Most Bookmarked Resources - Extended date range to all resources (it previously was limited to the top 10 resources)
8. Tops - Most Comments - All Time - Extended date range to all resources (it previously was limited to the top 10 resources)
9. Tops - Most Searched Terms - Fixed a bug that sometimes caused duplicate rows
10. Tops - Most Viewed Resources - Added “catalog” type category to the resource_type variable
11. Tops - Pageviews By Resource and Agentid - Added “catalog” type category to the resource_type variable

Updated Tables - For single-tenant

1. Events - Dataset or Project Views By Org - Name changed (from “Events - Views by Org”) and column name “dataset_views” changed to “views”
2. Resources - Org Owned Database connections - Added column “owner”
3. Tops - Bookmarks - Extended range to all users (it previously was limited to the top 10 users) and added column “displayname”
4. Tops - Dataset Creation - Extended range to all users (it previously was limited to the top 10 users) and added column “displayname”
5. Tops - Most Viewed Resources - Added “catalog” type category to the resource_type variable
6. Tops - Pageviews By Resource and Agentid - Added “catalog” type category to the resource_type variable

Watch this month's Gra.fo Round Up to learn more about our recently released features!

1. Drag and drop relationships

Relationship arrows can now be repositioned with a drag and drop action.

2. Export concept as PNG

Download a snapshot image of a concept and its immediate context.

3. Embedded image previews for link to concept

The link to concept feature now includes rich image previews.

4. Composite graphs

Link multiple Gra.fo documents together into one workspace. Separate complex models into subgraphs or extend reference documents that are used in multiple projects.

Round Up

Composite Graph Demo

For customers leveraging data artifact lineage, the resource items are now interactive. You can click through the icons to view the respective resource pages.

Updated metrics tables/reports have arrived on September 16, 2021! Some reports may take 24-48 hours to reflect the new data after deploy due to sync timing.

New Tables (multi tenant & single tenant)

• Resources - Dataset Files - A detailed listing of all (currently existing) files residing in datasets.
• Tops - Engagement - A list of users ranked by key engagement metrics.
• Resources - Live Metadata Assets Created - By Day - A long form series of counts of metadata assets created by date.
• Events - Dataset Activity - By Day - A fact table containing dataset activity measurements, aggregated by UTC-based calendar day.
• Events - Metadata Assets Activity - By Day - A fact table containing metadata assets activity measurements aggregated by UTC-based calendar day.

Updated Tables (multi tenant & single tenant)

• Membership - Current - By Org - Added new columns for email address, user display name, org-level authorization settings, org-level visibility settings and date of most recent update to authorization settings.
• Events - Searches - Last 90 Days - Fixed a bug that caused the counts of search results to be capped at 10.
• Events - Downloads (previously Events - Downloads - Last 90 Days) - Extended the timeframe to all-time; added new columns for file labels and user displayname. 

Base platform data updates (single tenant only):

• DOWNLOADS - new columns: (type, filename, filelabels) providing information about file downloads.
• DAILY_DWEC_ASSET_FACTS - new fact table providing measurements of metadata asset activity by date.
• FILES_DATASET_DIM - new dimension table providing information about files residing in Datasets.

The data.world public API supports several options for programmatically making updates to resources on the platform. PATCH is a method for making partial updates to individual records, such as adding tags, changing a description, or modifying a title.

In the next two weeks, we will be making a change to the way PATCH endpoints modify list values. We outline these changes below.

Existing Merge Behavior

Lists are merged with existing values on PATCH requests

1. A dataset has tags: [tag A , tag B] 
2. A PATCH request is sent to /datasets/democorp/my-example-dataset with body: { "tags": [ "tag C", "tag D" ]  }
3. The dataset is updated to reflect tags: [ tag A, tag B, tag C ,tag D]
4. A PATCH request is sent to /datasets/democorp/my-example-dataset with payload: { "tags": []  }
5. No change is applied and the tags remain: [ tag A, tag B, tag C ,tag D]

New Replace Behavior

Lists replace existing values on PATCH requests

1. A dataset has tags: [tag A , tag B] 
2. A PATCH request is sent to /datasets/democorp/my-example-dataset with body: { "tags": [ "tag C", "tag D" ]  }
3. The dataset is now updated to have tags: [ tag C ,tag D].  tag A and tag B have been removed.
4. I send a PATCH request to /datasets/democorp/my-example-dataset with body: { "tags": []  }
5. The dataset has been updated to remove all tags.

Why we are making this change

Today, PATCH can be used to add, modify, or remove fields for all non-list values. With the current merge logic, items can only be appended to list values using PATCH. As a consequence, if you want to remove or reorder the items in a list, you must use the PUT method, which does not support partial updates and requires a full overwrite of the existing record. The new logic to overwrite list values will allow users to make partial updates to records that remove or modify the order of items in the list without needing to modify the entire record.

This new logic primarily impacts tags, file labels, collections, and multi-select custom metadata fields.

This week, our search page got a face lift and we unveiled new tools for searching on data.world.

Explore data.world's rich advanced search syntax with the Search Builder tool on our main search page. This friendly form helps you construct more complex searches with multiple filters, logical operators, categories, and custom metadata fields. The Search Builder can be accessed by selecting the "Advanced" option above the filters list on the main search page.

This release also includes changes to our main search page. You'll notice a new layout on the All Results tab that shows the top 3 search hits by type for your term. This tab now shows more results per page and gives users a high level overview of the types of resources they can find on the platform. Hover over the circular "i" icon for more details about the result. More targeted results can be viewed on the Resources tab. You'll also notice changes to the category tabs at the top of the page. Resources, Organizations & People, Comments, and Columns each have their own tailored search experience.

The new search experience is available today for select users and will be available for all users early next week.

We're happy to announce the beta release of our new Postgres Proxy support. Our goal is to allow any BI and data science tools which support PostgreSQL, to connect to directly with data.world. No 3rd party integration support required!

data.world provides a federated engine to query data from multiple data systems simultaneously, at the source. By using Postgres Proxy, it’s now easier than ever to extend these capabilities to your favorite analysis tools for quickly accessing and creating value from data.

To connect to data.world using the proxy, simply create a new PostgreSQL connection, configured as follows:

host: postgres.data.world
port: 5432
user: {your data.world user id}
pass: {read/write token}
db:   agentid/datasetid

You can find your read/write token in the user settings. If you have any issues or questions, don't hesitate to reach out.

Note: for single tenant customers, set host to postgres.{site}.data.world.

For enterprise customers, updated metrics have been released today to your Usage and Governance Reporting (ddw-metrics-*) dataset to address some minor bugs and performance improvements.

Potential observable changes:

• For both single-tenant and multi-tenant customers, some reports were not reflecting de-provisioned user accounts. With this fix, multi-tenant customers may find a slight increase in counts in Visits - Return Visitors - Daily, Visits - Return Visitor Days, Visits - Unique Visitor Days and Visits - Unique Visitors - Monthly now properly reflecting de-provisioned user accounts in addition to active user accounts. Single-tenant customers may find a slight increase in Visits - Unique Visitors Daily and Visits - Unique Visitor Days.
• For multi-tenant customers, under certain circumstances some reports could duplicate-count users that were members of multiple sub-organizations. With this fix, you may find a slight decrease in counts in Membership - By Date, Visits - Adoption Daily and Visits - Avg Visits Weekly.

If you have any questions or concerns, please let us know at support@data.world or via your customer success representative.

In August, we plan to release a series of improvements to our search page including: 

• Category tabs to replace the result type dropdown
• Collapsible filter groups for the left sidebar
• A new discovery-driven All Results Page to highlight a more comprehensive set of result types for broad searches
• A new Advanced Search Builder utility for more complex searches that take advantage of boolean operators and our advanced search syntax

We're investing in navigation enhancements across the site for community and enterprise users. This week, we plan to roll out a beta breadcrumbs feature to help you get back to your list views from your resource pages. This first beta release will include easy access to your organization page from metadata catalog resources, quick filters to navigate back to similar items in your catalog, and basic hierarchical navigation for things like tables and columns. Additional hierarchical navigation support is planned for the coming months.