Harvest your Amazon S3 bucket and object metadata with our newest Collector

Introducing our newest metadata collector: Amazon S3 🎉 

The Amazon S3 Collector catalogs buckets and objects, allowing you to quickly search and discover your data. This new collector harvests metadata about buckets and objects, including the Region, Version State, Size, Last Modified Data, ACL Owner, Grantee and Grant Permission, amongst others (see the full list in the documentation). 

Inside the data.world platform, users will be able to view the relationships between S3 buckets and Objects, enhancing data discoverability. Using our configurable UI, you can display which pieces of metadata are most important to you, such as ACL Permission or S3 Metadata Keys and Values.



Learn how to use the new Amazon S3 Collector in our documentation, or please reach out if you have questions!

Profiling: a new kind of metadata

With the new year comes new features! We are pleased to launch our newest metadata capability: data profiling. This new feature creates metadata describing summary statistics for columns when a collector is run.

These summary statistics will help you understand and trust your data by providing a quick look at the data. For instance, viewing stats like the minimum and maximum values shows the shape of the data, allowing you to know quickly if your data is as expected. 

How can you create profiling metadata? This feature is currently available via the Snowflake, SQL Server, PostgreSQL, and Redshift collectors with more collectors on the near horizon. There are three optional commands that can be used during the collector run to generate the profiling metadata: 

--enable-column-statistics  description: enables harvesting of column statistics

--sample-string-values  description: enables harvesting of histograms for columns containing string data

 --target-sample-size  description: controls the number of rows sampled for computation of column statistics and string-value histograms

You can read more about these commands on the following collector documentation pages:

Enhance your Data Governance with Snowflake Tag and Policy harvesting

We are very excited to announce our newest metadata collection feature: Snowflake Tags & Policy harvesting! 

The Snowflake collector can now harvest Snowflake object tags, Snowflake tag-based masking policies, and Snowflake row access policies. This new feature will enhance your data governance experience by allowing you to see if a tag or policy is applied to a table or column coming from Snowflake.

For instance, here is a screenshot from the data.world catalog showing how a Snowflake Tag-Based Masking Policy has been applied to sensitive data columns: routing numbers, bank name, and bank account number. In this view, you can also see the associated Tag (Classification:confidential), and technical details about the Policy, like the Policy Body which explains how the Policy works.


How can you use this new feature? There are 2 optional commands for harvesting this information within the Snowflake collector run. Read more about it in the Snowflake Collector documentation.

Stay tuned for more exciting governance features in the coming months!

Collection Access Control is here!

Big news today! 🎉

You asked for more granular control of your data catalog and we listened. We're excited to introduce Collection Access Control. This NEW Feature is going to help you scale your Agile Data Governance program by providing more granular ways to control the access and management of your data catalog.



Collection Access Control provides role-based control to your metadata resources by collection, helping you target who can see and edit the resources in your catalog.

Read more about how to manage collection access on our documentation portal.

Create Custom Resources in the UI

Custom resources play an important role in a data catalog's ability to accurately represent your company's metadata. Users need to be able to add resources that aren't directly from data sources to give a full picture of their data landscapes. Now, users can create these custom resources directly from the UI and manage them like any other resource.

After designating which resources should have this feature enabled in your metadata profile, users will be able to access these resources in the "Other resources" section in the "New resource" dropdown (in the example below, the custom resources are "Bank account" and Credit card").

For more information, refer to the documentation.

Harvest Data Observability with Monte Carlo Collector!

We're pleased to announce our 2nd-generation Monte Carlo collector is now live for beta customers! Monte Carlo is a Data Observability Platform that lets users know about data problems (like broken pipelines), so they can proactively resolve issues. 

The Monte Carlo collector harvests both Incidents and Monitors, which inform users about the issue, when it happened, and where it happened.

For example, users can view relevant information about Incidents and Monitors, like Status, Count, the Date Created, as well as Owner and Severity. You can also open the specific Incident in Monte Carlo directly from the data.world platform. 

You can read more about the newest Monte Carlo collector in our documentation

Existing customers, please reach out to your data.world representative to learn more about becoming a beta user.

Configure browse cards for your organizations

Quick update! The browse card - a configurable, quick navigation card that directs end-users to a curated starting point - was such a hit with our private instance Enterprise customers that we've extended it so you can configure one for each of your Organization profile pages. Woohoo!

Read our Implementation Guide to get started.

Glossary Bulk Import and Bulk Edit

Introducing our newest feature: bulk import and edit for glossary terms!

Catalog administrators can now bulk upload glossary terms for faster onboarding and enrichment, and make bulk edits to ensure your catalog is always accurate.

This feature enables the following workflows:

  • upload glossary terms using a template custom to your catalog's metadata profile

  • download a spreadsheet of all current glossary terms

  • edit information, add new terms, delete terms in exported spreadsheet

  • upload updated spreadsheet to application

  • preview changes made in spreadsheet before applying changes

For more information, refer to the full documentation here.

New Connection Manager Data Source: Tableau

We now support metadata collection from Tableau via Connection Manager. Previously, users were able to collect metadata via the data.world Collector, but now users can perform this task in-app.

Both Tableau Online and Tableau Server are supported, as well as the following connection types: Direct connection (inbound), SSH tunnel (inbound, preferred).

For more information, refer to the documentation here.

New configurable browse card

We've been super busy over the last few weeks, and we're excited to share that we've officially launched a new feature called the browse card, a configurable, quick navigation card available for private instance Enterprise customers.

This configurable card shows up on your Action Center home page and helps you direct your users to important resources, organizations, collections, and other links - giving them a curated starting point. Read our Implementation Guide to get started or reach out to your Customer Success representative if you need to upgrade your catalog plan.



Show Previous EntriesShow Previous Entries