Here is a roundup of some of the key features we released last month, February 2021 -- Looker metadata support, custom metadata field default edit behavior improvements, Gra.fo enhancements, and dataset sync webhook notifications.

Check out recent UX improvements to Gra.fo, our visual knowledge graph modeling tool. 

1. To quickly zoom out to see the full graph, use the keyboard shortcut:
   OPTION (ALT) + SHIFT + 0
2. To zoom to a specific concept:
   OPTION (ALT) + SHIFT + click concept
3. You can now deeplink directly to a concept in the graph using the Copy link to concept option in the concept menu. Watch the video to see how we use this feature to move fluidly between a data.world catalog and Gra.fo.

Gra.fo is a companion product to data.world with a 30 day free trial of premium team features. Visit Gra.fo to learn more, or reach out to help@data.world.

An update is coming soon for the following metrics tables.

Events - Pageviews - Last 90 Days: This table now includes all types of pageviews, whereas previously it was filtered to only include dataset and project views. In addition, the window has been extended from 30 to 90 days.

Events - Queries: This new table is a detailed audit of query events. This was previously already available for multi-tenant customers. Going forward, it will also be available for single-tenant or virtual private customers. More details are included in the data dictionary markdown file. 

Resources - Datasets: This table contains a new column called “notificationsEmail” which includes any additional email address associated with the notifications for the dataset, as set in the dataset settings.

Visits - New Users By Month: This table logic has been updated to be consistent with other metrics, namely Visits - Unique Visitors To Date and Visits - Unique Visitors - All Time List.

We're proud to announce a new metadata collector is now available for enterprise customer use in private beta -- Looker (business intelligence). Let us know if you'd like to get your hands on the early version of this collector.

As we get closer to general availability, we'll be releasing more information on these exciting expanded metadata and discovery capabilities!

Here is a roundup of some of the key features we released last month, January 2021 -- broader resource support for recently viewed resources and auto suggestions from the search bar, collection search filter/facet, collection management API, and enhanced SQL support for windowed aggregations.

data.world now sends 2 additional webhooks: Dataset Sync Failure and Dataset Sync Recovery Success. 

When a dataset within your org fails to sync, data.world will immediately send a webhook message to the configured URL with details about the failure, including error code and error messages from the downstream source.

With those sync failure details, you could configure a ticket to be created in your ticketing system, trigger an alert for a dev-ops team, or simply just make the team aware of the issue.

Natively, data.world will format the webhook payload to post messages in Slack when it recognizes the configured webhook is Slack, which is shown in the demo video below.

Demo

Screenshots

Technical Details

syncTask.status.success is triggered only after the recovery from a failure. 

Example payload, in JSON:

{
  "datasetid": "sync-demo",
  "event": {
    "type": "SUCCESS",
    "task": "state_table.csv",
    "created": "2021-02-04T06:37:06.936Z"
  },
  "eventType": "syncTask.status.success",
  "resourceUrl": "https://data.world/texas-whiskey/sync-demo/file/state_table.csv",
  "resourceType": "URL",
  "resourceOwner": "texas-whiskey"
}

 syncTask.status.failure is triggered on the first failure for that file. When it recovers, data.world will send one syncTask.status.success

Example payload, in JSON:

{
  "datasetid": "sync-demo",
  "event": {
    "type": "FAILURE",
    "task": "state_table.csv",
    "created": "2021-02-04T06:35:17.945Z",
    "error": "HTTP/1.1 404 Not Found"
  },
  "eventType": "syncTask.status.failure",
  "resourceUrl": "https://data.world/texas-whiskey/sync-demo/file/state_table.csv",
  "resourceType": "URL",
  "resourceOwner": "texas-whiskey"
}

Have you ever need to create a 7-day trailing average in order to smooth out a graph? How about compare a value to the same value in the previous row?

Windowed aggregations are a powerful way to perform complex analysis, such as this, without leaving the comfort of the SQL. These functions operate on a defined group, or "window", of rows to return an aggregated value for each row. Window aggregations are supported for both uploaded tabular data, as well as live tables (where support exists in the connected database).

Note: this feature is currently in Beta. Please contact support with any issues.

Keep your eyes open for an exciting new feature in upcoming enterprise releases.
Soon, our standard search behavior will be expanded to include custom metadata associated with your catalog resources. This feature will enable users to execute generic searches such as "Smith" and find resources where "Jane Smith" is listed as "Steward" (a custom metadata property) for a given resource. This feature will be accessible to enterprise customers and will include public API support.

 🚨 Default behavior change coming soon! 🚨

data.world offers the ability to define custom metadata fields on resources like datasets, projects, glossary terms, dashboards, and metadata tables. Custom metadata field types include free text fields, drop-down fields, and multi-select hierarchical fields.

To provide a focused user experience while editing custom metadata, each field had a configuration option to either show the field during the editing process, or else put the field behind a drop-down, requiring a user to find the field in the drop-down before setting its value. The default behavior was to hide the field in the "Add new field" drop-down (pictured below), unless it was flagged as a primary field.

While we typically avoid changing the default behavior of features with widespread adoption, conversations with customers has led us to flipping the default — all fields will show up during the editing experience, unless they are now flagged a secondary field. Only if they are considered a secondary field will they ever be behind the "Add new field" drop down.

The benefit of this change is that users will more easily see all the fields available to edit or provide crowdsourced suggestions for, including fields that are empty and don't have a value set yet.

The edit experience, along with the "Add new field" drop down, in action:

Bottom line - this will result in more fields showing up by default when you click Edit (they won't be hidden under the 'Add new field' drop-down) - unless you mark them as secondary in your metadata configuration (which will hide them under the drop-down).

If you have any concerns or questions on how to implement the secondary setting, including if you'd like to implement it proactively prior to the feature being rolled out, please reach out to your customer success representative or support@data.world and we'd be happy to assist!

Find what you are looking for from anywhere in the application.
Look for new resource types added to the autosuggest feature of the main search bar! Queries and metadata resources, such as glossary terms and analysis records, have been added to the top level search bar. You can now search for these resources from any page with the search bar header. This feature previously supported dataset, project, and user search. This feature is live for community and enterprise users.