We are excited to announce the launch of new features and latest improvements:
- Cloud Collectors - configure and run collectors hosted by data.world NEW
- Support for Snowflake Data Quality - collect and catalog Snowflake Data Metric Functions (DMFs) NEW
- Bulk operations UX improvements - streamlined bulk enrichment workflow IMPROVED
- Enrichment and discovery UX improvements - more context and default sorting IMPROVED
Read the sections below for full details on each new feature!
NEW Introducing: Cloud Collectors!
We are excited to announce the launch of Cloud Collectors, the newest way to collect metadata on data.world!
Now, you can configure and run collectors that are hosted by data.world with just a few clicks! This feature not only provides a no-code way to start bringing metadata into your catalogs faster, it also has robust functionality around scheduling and monitoring to make setup more transparent and seamless. If you have cloud-accessible data sources that you're ready to bring into your catalog, this feature is for you!
👩💼 How can I use Cloud Collectors?
Users with Admin access will see a new option in the collector setup wizard that says "Cloud."
Once you enter your source information, you will be able to set a custom name for your collector configuration, and set a schedule for how frequently the collector should run.
After a collector completes, you will see the metadata and resource types that were collected, as well as the source information you entered while setting up the collector. Here you will also find what might have gone wrong if the collector run failed, and you'll have the ability to cancel the run as well.
You can view all of the collectors you have set up, whether they are from collectors that you host or Cloud Collectors, on the Metadata Collection tab. From here, you can view, edit, and delete collector configurations. And if you're setting up multiple collectors for one source with the same credentials, try the "Duplicate Configuration" button to quickly set all of them up.
For a full list of supported sources and more details on the feature, please refer to the documentation here.
NEW Announcing support for Snowflake Data Quality
We are thrilled to introduce an exciting addition to our existing Snowflake collector – support for Snowflake’s brand-new Data Quality feature, currently available in private preview. This enhancement empowers users to elevate their data quality assessment to new levels.
Key Features:
📊 Collect and catalog Snowflake Data Metric Functions (DMFs): Users can now measure the quality of their data using Snowflake’s powerful "data metric functions" (DMFs) and catalog this context with data.world. Example DMFs include Null Count, Unique Count, and Freshness – providing comprehensive insights into the health of your data.
🔍 Find and understand data quality metrics: The DMFs and observations (recorded metrics) are seamlessly integrated into resource pages on your data.world platform and are also presented as Hoots associated with Snowflake tables and views. This user-friendly interface makes it easy for individuals across your organization to discover and understand data quality metrics effortlessly.
Why Snowflake Data Quality & data.world?
🌐 Compliance & Consistency: In today's data-driven landscape, ensuring compliance and consistency is paramount. This Data Quality feature & integration help you meet these standards by offering real-time insights into critical data metrics.
🔒 Build Trust: Trust is the foundation of effective data utilization. This Data Quality integration helps users to trust their data by bringing metrics related to freshness, blank values, and inaccuracies to the catalog and everyday tools, such as Tableau and Power BI, via Hoots.
Who Benefits?
👩💼 Data Stewards, Engineers, Admins: Empower your data stewards and technical teams by providing them with a tool that gives immediate insights into the current state of their data based on specific metrics.
🚨 Data Consumers: With Hoots, you can identify and take swift action on tables and views that require attention, ensuring data quality monitoring is seamlessly integrated with considerations for cost, consistency, and performance.
Experience a new era of data quality and reliability with data.world’s support for Snowflake's Data Quality today!
Note: Snowflake Data Quality is an enhancement within the existing Snowflake collector, and is currently available to Snowflake Private Preview customers. ❄️🚀
Using Hoots, users can quickly see data quality issues, like duplicate data, and easily fix the errors.
Improvements and Enhancements
IMPROVED Improvements to Bulk Operations UX
Bulk operations are a crucial part of keeping a catalog updated and accurate. We're excited to announce some improvements that will streamline and accelerate bulk operations such as bulk editing tags and attributes and bulk moving resources between collections.
First, we have consolidated these operations into a single menu for each place you can initiate a bulk operation (the Glossary tab, the Resources tab, and the Collection Contains tab). Now you can Quick edit, Add resources to collections, and Export/Import resources from all three locations.
Next, we've added the granular selection experience, that previously existed only in Quick edit, to the Export/Import spreadsheet flow as well. This is available on all three entry points (Glossary, Resources, Collections), which should significantly reduce the time it takes to make changes via the spreadsheet option.
Finally, we've simplified and clarified the experience around moving resources between collections. Previously this experience only existed within the Quick edit flow, but now you can select 'Add to Collections' or 'Move or Add Collections' to access this functionality. From the Glossary and Resources tab, you'll be able to add resources to one or multiple collections, and from the Collection tab (example below), you'll be able to add resources to one or multiple collections, or move resources from one or all collections to one or multiple collections.
With these improvements, administrators and curators will be able to perform bulk operations on resources much more quickly. For more information, please refer to the documentation for bulk editing resources here and for bulk editing glossary here.
IMPROVED Added context in various search experiences
The suggested search dropdown now has more context, including the list of collections, owning Organization or User profile, and more. We’ve also added more context to the search experience when a user is relating one resource to another. This added context makes it easier to see and understand what has already been added.
IMPROVED Default sorting improvement + column index sort
We’ve provided a default sort experience that makes scanning the related, contained, and column resources faster. We also added column index as a sort option so users can understand the original column order from the database.
IMPROVED Expansion of the Summary field
The Summary field is now available on all resource types out-of-the-box. The field is available on all Types without the need for configuration.
IMPROVED Rich Text Editing without Markdown
Multi-line fields on catalog resources support Rich Text for more engaging and understandable content, and now these fields can be edited in a What-You-See-Is-What-You-Get (WYSIWYG) user experience rather than users having to create and edit content using Markdown.
Markdown editing is still available for users that prefer it, but now more data owners and users can create compelling rich text content.
