We've added a helper utility to our public API to make it easier to discover the configurable custom metadata properties your organization supports when interacting with your catalog through the API. Head over to our API documentation to give the metadata/properties endpoint a try.
data.world not only supports metadata collection, but can also power federated queries of data sources such as Snowflake, BigQuery, SQL Server, and many others. Now in addition to our UI, you can use our API to manage these virtual connections and tables.
Head over to our API documentation to learn more about configuring virtual connections and adding virtual tables to your datasets with the public API.
Need some quick feedback on your collaborative modeling project? Want to share your graphical view of the world with your customers or your team?
You can now share your Gra.fo model documents without requiring your audience to have a Gra.fo account or an individual invitation. Use the "Get link" option of the share menu to grant read-only access to your document to anyone you've shared the link with.
One of the most powerful features of the data.world catalog is the ability to enrich your catalog resources with custom metadata that's unique to your business.
We are pleased to announce that we've rolled out the first of several improvements to support search matches on custom metadata fields for your catalog resources.
This improvement expands the fields we match against for free text searches to include any custom metadata fields you have configured in your catalog as text or selection fields. This feature empowers your end users to search for resources by the terminology and categorizations that mean the most to your business.
In the example above, verified by and data steward are custom metadata fields defined in our catalog for tables. A search for sarah smart now yields matches where she is listed as the data steward or the person who has verified the data, in addition to any existing matching fields like owner.
Tip: You can perform more precise searches against custom metadata fields with our advanced search syntax. In the example above, a search for
metadata:"data steward:sarah smart"will return filtered results where Sarah Smart is listed as the Data Steward.
Look for upcoming releases to further support boolean and IRI-based metadata searches.
🚨 Default behavior change coming next week 🚨
We have recently discovered that when executing queries, there are some cases where our DATETIME columns contain timezone information, and other cases where they do not. This is primarily an issue that arises with columns containing date/time information in uploaded files (we do not see this with live tables). We have decided to address this inconsistency. Starting next week, query result columns of type DATETIME will no longer contain timezone information, while columns of type DATETIMESTAMP will always contain timezone information.
The impact of this change shouldn’t be significant, and most users will see no change. However, if you have queries across ingested data which aggregate on DATETIME columns, or do DATE_ADD() style calculations, you may notice differences in your results depending on your current timezone.
If you are impacted by this change, here are some ways to clarify your intent w.r.t. timezones:
CAST the resulting column to a DATETIMESTAMP to force timezones, or DATETIME to strip timezones (documentation)AT_TIME_ZONE() to explicitly state your timezone (documentation)DATETIMESTAMP or DATETIME (documentation)Note: If timezone information is desired, but not defined, UTC is assumed.
Please contact support@data.world with any questions or concerns. As always, we’re happy to help.
data.world is very excited to announce our new deep brain integration.
Now data consumers simply need to think about what data they want, and data.world will return governed, curated data. It also supports cataloging of business terminology straight from subject matter experts.
When we originally envisioned the feature, our design inspiration was to provide an "easy button." However Jon Loyens, co-founder and CPO, famously then said "what if there was no button at all?"
A future release will support agile data governance workflows, such as data access approvals. Integration is quick and relatively painless, though upgrades require a bit of effort and minor outpatient surgery.
Head over the Gra.fo API Documentation to learn more about how to use the Gra.fo public API to export your model as TTL or OWL. The export API is designed for use with build scripts, version control tools, integrations, or to upload your model into other tools, like data.world.
Fun fact: Ever wonder why our adorable mascot, Sparkle, is an OWL? Now you know!
Looking for the API Documentation? We've added a link to the user menu in Gra.fo:
Gra.fo supports several additional export formats from the document page. These options can be found under the File menu.
Connection Manager now supports collection of metadata from the platform user interface, enabling a faster, more consolidated way to catalog your data. We currently support collecting metadata from a number of different database sources, and we will be expanding the list of supported sources over the coming months.
View metadata task documentation
Here is a roundup of some of the key features we released last month, February 2021 -- Looker metadata support, custom metadata field default edit behavior improvements, Gra.fo enhancements, and dataset sync webhook notifications.
We're proud to announce a new metadata collector is now available for enterprise customer use -- Presto (distributed SQL query engine). Presto is a popular way to query data across a variety of different data sources with a single, optimized interface. Metadata collected includes database, schema, table, view, and column information.
