Use the "None of the following" option in the Advanced Search Builder to exclude search results that match a specific value. This feature works with the logical NOT operator in our Advanced Search Syntax.
Expand and collapse controls are now available for the search filters sidebar to help you quickly locate available filtering options.
The Organization Profile Page is getting a facelift this week to offer more intuitive navigation, stronger support for custom catalog types, and richer discovery features.
Browse the Overview tab for quick links to different resource categories in your catalog. The quick link tiles will take you to a filtered presentation of the new Resources tab. This view operates much like the main search page with support for facets and advanced search syntax, all scoped to your organization's resources.
Searching for open data? Community organizations will now also have these search and filter options available on the new Resources tab.
Another neat feature for our customers leveraging data artifact lineage! In addition to being interactive, the resource items now display a summary card when hovering. You can preview metadata at a glance and click through to the collection, individual tags, or to the resource itself.
Updated metrics tables/reports have arrived on October 18, 2021! Some reports may take 24-48 hours to reflect the new data after deploy due to sync timing.
Data dictionary has been updated to reflect the latest updates as well.
Most analysts trying to find answers to business questions aren’t searching for tables and columns directly. What they are actually looking for is contextual information that accelerates time to business impact for data. data.world Concept Cards will change the way data consumers access data by providing a unique search experience no other catalog provider does or can do without the backing of a knowledge graph.
Concept Cards are a feature on data.world’s near-term roadmap to help users discover related people, resources, and other supporting information we can obtain from the knowledge graph about a given search topic. If there are suggested actions that can be taken for the topic itself or for related resources, access to those actions is surfaced directly in the search results.
These cards become a jumping off point to browse and discover new things on the platform that share something in common with the search topic of interest. We see these Concept Cards as the first of many intelligent recommendations we can make by harnessing the power of the knowledge graph.
Querying data in its current state is the most common data catalog use case, but there are times when it is necessary to compare previous versions of datasets, metadata, and lineage. data.world SQL and SPARQL Time Travel allows customers to view changes across metadata and data and even query historical data sources.
The new feature provides granular insight into audit trails and analysis of data that is snapshotted across time. You can search both ingested data sources and Snowflake virtual tables for previous states of data. Being able to analyze previous versions of a dataset, even simultaneously with the current version of a dataset, enables flexible analysis across various time scales – review data month-over-month, year-over-year, etc.
In data.world, your metadata is also data and therefore fully queryable and reportable. You can compare previous versions of your metadata with current versions in order to understand how your systems and schemas are changing. See new columns, new column names, sensitive data that recently appeared in a field that wasn't there previously, and much more.
Supported operations include previous version, number of versions back (tip-N), specific timestamp, and offset.
Example: SQL Time Travel Query
Example: SPARQL Time Travel Query
A key aspect of data compliance is knowing where sensitive data lives and applying classifications that relate to policies that inform business processes for proper tracking and management. Identifying sensitive data, applying these policies, and reporting on this information can be an extremely time consuming and error-prone task if attempted manually.
data.world’s Sensitive Data Discovery automates discovery and classification, making it easier for enterprise customers to identify sensitive data and take action on it within the catalog.
Scan – Use advanced machine learning to identify sensitive data types like email addresses, names, ID numbers, locations, protected health information, and 40+ additional data types identifiable out of the box.
Classify – Apply policy classifications, tags, and statuses such as Restricted, Personal Information, US Only, etc. These classifications help maintain the integrity and confidentiality of your data. They are driven by your scan results and other metadata, as dictated by your unique business logic and terminology.
Take Action – Report and audit sensitive data types and policy classifications across your data landscape, understand how it changes over time, and drive better compliance and governance in your organization.
Integrate – Leverage Sensitive Data Discovery metadata as part of your broader metadata orchestration strategy with APIs and bulk export. Our open and extensible platform makes it easy to plug in your broader ecosystem of additional Sensitive Data Discovery tools and platforms for even greater governance capabilities.
Resource page example
Search results example
If you are an existing data.world customer and would like to be included in the private beta, reach out to your Client Success Director for more information.
For customers leveraging data artifact lineage, the resource items are now interactive. You can click through the icons to view the respective resource pages.
Updated metrics tables/reports have arrived on September 16, 2021! Some reports may take 24-48 hours to reflect the new data after deploy due to sync timing.
New Tables (multi tenant & single tenant)
Updated Tables (multi tenant & single tenant)
Base platform data updates (single tenant only):
The data.world public API supports several options for programmatically making updates to resources on the platform. PATCH is a method for making partial updates to individual records, such as adding tags, changing a description, or modifying a title.
In the next two weeks, we will be making a change to the way PATCH endpoints modify list values. We outline these changes below.
tag A , tag B] PATCH request is sent to /datasets/democorp/my-example-dataset with body: { "tags": [ "tag C", "tag D" ] }tag A, tag B, tag C ,tag D]PATCH request is sent to /datasets/democorp/my-example-dataset with payload: { "tags": [] }tag A, tag B, tag C ,tag D]tag A , tag B] PATCH request is sent to /datasets/democorp/my-example-dataset with body: { "tags": [ "tag C", "tag D" ] }tag C ,tag D]. tag A and tag B have been removed.PATCH request to /datasets/democorp/my-example-dataset with body: { "tags": [] }Today, PATCH can be used to add, modify, or remove fields for all non-list values. With the current merge logic, items can only be appended to list values using PATCH. As a consequence, if you want to remove or reorder the items in a list, you must use the PUT method, which does not support partial updates and requires a full overwrite of the existing record. The new logic to overwrite list values will allow users to make partial updates to records that remove or modify the order of items in the list without needing to modify the entire record.
This new logic primarily impacts tags, file labels, collections, and multi-select custom metadata fields.
