Unity Catalog Row Filters, Column Masks and ABAC Policies
Introduction
This lesson completes the Governance and Security section of the official exam outline (about 15% of the scored questions). The Unity Catalog tables and privileges lesson established object-level grants. This lesson applies Unity Catalog row filters, column masks and ABAC for the Databricks Data Engineer Associate exam. At the level of whole objects, a principal can or cannot read a table. Real organizations need finer control. A regional sales manager should see only the rows of their region; a support agent should see the last four digits of a card number but not the whole value; an analyst should see the data but never the email addresses.
Unity Catalog offers three tools for this fine-grained control: row filters, column masks and, at scale, attribute-based access control (ABAC) policies driven by governed tags. Row filters and column masks are defined per table. ABAC applies the same logic automatically to every table that carries the right tags, across a whole catalog or schema. Databricks announced ABAC row filtering and column masking policies, governed tags and data classification as generally available in 2026; its documentation recommends ABAC for consistent rules across many tables, and per-table filters and masks when you need logic specific to one table or have not adopted ABAC yet.
Learning objectives
- Explain the difference between object-level privileges, row filters, column masks and ABAC policies.
- Write a row filter function and attach it to a table.
- Write a column mask function and attach it to a column.
- Use
is_account_group_member()to implement identity-based logic. - Explain ABAC: governed tags, policy scope, principals, conditions and the policy functions.
- Choose between a dynamic view, per-table filters/masks and ABAC.
- Name the limits and operational points to remember (privileges, performance, and what consumers see).
1. Why object-level privileges are not enough
Imagine the table prod.sales.orders, with columns order_id, region, customer_email and amount.
| Requirement | Object-level GRANT SELECT |
What is needed |
|---|---|---|
| EU managers see only EU rows | Cannot express it | Row filter |
| Everybody sees the table but the email is hidden except for support | Cannot express it | Column mask |
| All columns tagged as personal data are masked in 400 tables | Would need 400 separate definitions | ABAC policy with governed tags |
Before these features, engineers created dynamic views per table, with a CASE expression or a WHERE clause using is_account_group_member(). A dynamic view remains useful for curated or joined datasets; finding a base table does not grant access to it. Restrict base-table SELECT when a view is the intended security boundary. A view can combine multiple tables rather than requiring exactly one view per table. Row filters and masks are attached to the table itself, so supported Unity Catalog query paths enforce them. This is not protection against independent cloud-IAM access to raw files, nor a guarantee that every external API or older compute mode supports the table.
2. Row filters
A row filter is a SQL user-defined function (UDF) that returns a BOOLEAN. It is attached to a table and receives the values of one or more columns of each row. Unity Catalog evaluates it for every query on the table; rows for which it returns TRUE are returned, and rows for which it returns FALSE (or NULL) are hidden. The reader does not see the filter in the query: the rows are simply absent.
-- 1. The function
CREATE OR REPLACE FUNCTION prod.security.region_filter(region STRING)
RETURNS BOOLEAN
RETURNS BOOLEAN
RETURNS BOOLEAN
RETURN is_account_group_member('global-managers')
OR (is_account_group_member('eu-managers') AND region = 'EU')
OR (is_account_group_member('us-managers') AND region = 'US');
-- 2. Attach it to the table, naming the column(s) passed to the function
ALTER TABLE prod.sales.orders
SET ROW FILTER prod.security.region_filter ON (region);
-- Optional cleanup exercise only; this removes the attached protection.
-- Run separately from installation, after reviewing the access requirement.
ALTER TABLE prod.sales.orders DROP ROW FILTER;Points to remember:
- The function is a normal Unity Catalog object, so it can be stored in a dedicated security schema with its own, tightly controlled privileges.
is_account_group_member('group')returns true when the current user belongs to the account-level group; it is the preferred check in Unity Catalog.current_user()returns the user's identity and is useful for a mapping table (for exampleuser_regions), but a mapping table must itself be protected.- A table can have one row filter; the function may combine several conditions.
- The filter can also be given other columns, or a constant argument, with types compatible with the SQL UDF parameters. Keep it simple and efficient; identity checks intentionally depend on the query identity. Do not infer that arbitrary non-deterministic logic, complex subqueries or external calls are supported in every read/write operation.
- Users need access to the table (
SELECTand theUSEprivileges) as usual; the row filter then trims what they receive. Attaching/removing a filter on an existing table requires table ownership or both MANAGE and SELECT, plus EXECUTE on the function and parent usage privileges. Creating the function also requires the relevant schema rights.
Concrete example. A bank stores loans(loan_id, branch_id, amount, ...). Branch managers must see only their branch. The security team creates branch_filter(branch_id) that compares branch_id with a lookup in a protected mapping table security.user_branches using current_user(). A manager runs SELECT COUNT(*) FROM loans and gets only the loans of their branch; an auditor in the group audit gets everything.
3. Column masks
A column mask is a SQL UDF attached to a column. It takes the column's value (and optionally other columns of the row) and returns the value to display. The return type must match or be castable to the column's type; prefer an exact type match, so that queries and tools do not break.
CREATE OR REPLACE FUNCTION prod.security.mask_email(email STRING)
RETURNS STRING
RETURN CASE
WHEN is_account_group_member('pii-readers') THEN email
ELSE '[REDACTED]'
END;
ALTER TABLE prod.sales.orders
ALTER COLUMN customer_email SET MASK prod.security.mask_email;
-- Optional cleanup: removes the column protection; not an installation step.
ALTER TABLE prod.sales.orders
ALTER COLUMN customer_email DROP MASK;Members of pii-readers see maria@example.com; other permitted readers see [REDACTED], without exposing the domain or malformed original strings. These snippets assume the named schema, groups and STRING columns exist; test them on an isolated practice table.
A mask can use additional columns of the same row through USING COLUMNS:
CREATE OR REPLACE FUNCTION prod.security.mask_card(card STRING, country STRING)
RETURNS STRING
RETURN CASE
WHEN is_account_group_member('payment-readers') AND country IN ('FR', 'DE')
THEN card
ELSE concat('****', right(card, 4))
END;
ALTER TABLE prod.sales.orders
ALTER COLUMN card_number SET MASK prod.security.mask_card USING COLUMNS (country);Properties:
| Aspect | Row filter | Column mask |
|---|---|---|
| Attached to | A table | A column |
| Function returns | BOOLEAN |
Same type as the column |
| Effect | Hides rows | Replaces values |
| Number per object | One per table | One per column |
| Removing | ALTER TABLE ... DROP ROW FILTER |
ALTER COLUMN ... DROP MASK |
Masks are not encryption. They control what a query returns. Seeing raw values depends on the actual mask logic and applicable policy exemptions, not simply on being an owner or administrator. Principals able to change a table-level mask or its UDF can change the protection; separate those duties. Stored files are unchanged, so also restrict direct cloud access.
4. Dynamic views versus filters and masks
| Criterion | Dynamic view | Row filter / column mask | ABAC policy |
|---|---|---|---|
| Protection point | A view, with base access restricted by its own grants | The table itself | The table itself, by tags |
| Reuse | A curated view can join/reshape several tables | One function reused by many tables, attached manually | One policy for many tables automatically |
| Effort for 400 tables | Very high | High (attach to each) | Low (tag once, policy applies) |
| Typical use today | Curated, transformed or joined access without base-table grants | Table-specific logic | Consistent enterprise rules |
5. Attribute-based access control (ABAC)
ABAC row-filter and column-mask policies restrict permitted queries using attributes of the data (tags) and, in policies, of the principals (their groups), in addition to the object privileges that allow a query. Separate ABAC GRANT policies can grant privileges; the filter/mask policies taught here do not. In Unity Catalog the building blocks are:
- Governed tags: account-level tags whose name, allowed values and the permission to assign them are controlled centrally. Unlike free tags, a governed tag cannot be created or applied by just anyone; a data steward decides who may assign
pii = email, for example. - Policies: rules that say "for the objects matching this tag condition, apply this row filter or column mask function, to these principals, except those principals".
- Policy functions: the same kind of SQL UDFs as in sections 2 and 3, now referenced by a policy.
A policy is defined on a scope: a catalog, a schema or a table. It is inherited downward, so a policy at catalog level applies to all current and future matching tables below it. The condition uses the tags of the tables and columns, and the principals can be users, service principals or groups, with an EXCEPT list for those not concerned.
The following policy masks matching email columns for everyone except the privacy team using the documented SQL syntax. This example targets STRING columns directly tagged with the governed pii=email value. The mask UDF always redacts; exemptions belong in the policy rather than hidden group checks in the UDF:
CREATE OR REPLACE FUNCTION prod.security.redact_email(email STRING)
RETURNS STRING
RETURN CASE WHEN email IS NULL THEN NULL ELSE '[REDACTED]' END;
CREATE POLICY mask_emails
ON CATALOG prod
COLUMN MASK prod.security.redact_email
TO `account users`
EXCEPT `pii-readers`
FOR TABLES
MATCH COLUMNS has_tag_value('pii', 'email') AS email_col
ON COLUMN email_col;A row filter policy follows the same pattern, with ROW FILTER and a condition on the table, for example tables tagged domain = sales that have a column tagged region.
Operational facts:
- Creating a policy requires the right to manage it on the securable (the
MANAGEprivilege on the catalog, schema or table, or ownership) andEXECUTEon the function used. - Tagging the data is the key step: a column without the tag is not affected. Governance teams can use data classification to detect and tag sensitive columns. Native classifier tags such as
class.email_addressare not automatically the custompii=emailtaxonomy used here: configure the matching policy/tagging process explicitly. Column tags do not inherit from a table/catalog; matching column tags must be applied directly. Tag assignment requires ASSIGN on the governed tag and APPLY TAG on the object. - Policies are evaluated at query time through supported Unity Catalog query interfaces.
- Keep current scope boundaries clear: metastore-level policies and ABAC on views have separate Beta requirements, and ABAC DENY policies (Beta) currently target MANAGE ACCESS CONTROL. Do not treat a Beta DENY policy as a general DENY SELECT statement. Advanced identity-attribute functions have their own documented contexts and limitations; they are not needed for the group-based examples here.
Concrete example. A pharmaceutical group has 1,200 tables across 30 schemas. Compliance requires that every column holding a patient identifier is masked for everybody except the clinical-data team. Without ABAC, an engineer would attach a mask to each column, then forget new tables. With ABAC, the data steward tags the identifier columns with pii = patient_id (through a tagging/classification workflow configured for that taxonomy), one catalog-level policy applies mask_patient_id to every tagged column, now and in the future, and the clinical-data group is the exception.
6. Operational points
- Compute: table-level filters/masks support SQL warehouses, standard DBR 12.2 LTS+ and dedicated DBR 15.4 LTS+ with required serverless filtering. ABAC policies require serverless or standard DBR 16.4+, or dedicated DBR 16.4+ with fine-grained filtering enabled. Dedicated writes and specific DML patterns have additional version limits; verify them rather than treating all modes as equivalent.
- Performance: the function runs on the query path. Keep it simple: no heavy subqueries or external calls. A mapping-table lookup should be small.
- Downstream objects: for a supported view over an ABAC-protected table, base privileges are checked for the view owner while policies are evaluated for the querying user. For materialized/streaming refreshes, verify the pipeline owner/run-as context. A persisted downstream result does not automatically establish the desired permissions or identity-dependent protection on the target. Secure that object explicitly and test through the actual consumer interface.
- Testing: test with accounts in each group, and check the result of
SELECT *,COUNT(*), joins and aggregations, because a mask applied before an aggregation changes the aggregate. - Auditing: inspect query access and tag/policy changes through supported audit/system-table views. Test overlapping masks/filters: conflicting applicable functions can fail a query rather than selecting the last policy.
Worked example: protecting customer data in a shared orders table
Scenario. prod.sales.orders has order_id, region, customer_email, card_last4 and amount. Requirements:
- Regional managers see only their region's rows; the group
global-financesees everything. - Everybody except
support-tier2seescustomer_emailmasked. - The same email rule must apply to 80 other tables in the catalog
prod.
Step 1: Row filter on this table. Table-specific logic fits a per-table row filter.
CREATE OR REPLACE FUNCTION prod.security.region_filter(region STRING)
RETURNS BOOLEAN
RETURN is_account_group_member('global-finance')
OR (is_account_group_member('eu-managers') AND region = 'EU')
OR (is_account_group_member('us-managers') AND region = 'US');
ALTER TABLE prod.sales.orders SET ROW FILTER prod.security.region_filter ON (region);Step 2: Email masking for 80 tables. Use the unconditional redact_email UDF above, have the steward directly tag every email column with pii=email, and create the catalog-level policy with support-tier2 in its EXCEPT list instead of pii-readers. Do not reuse the earlier conditional mask_email as the policy function: it would also unmask pii-readers and violate this scenario. Do not attach that separate per-table email mask to the same column in this scenario.
Step 3: Grants. Make sure the groups hold USE CATALOG, USE SCHEMA and SELECT, because filters and masks restrict what is returned and do not grant access.
Step 4: Test. A user in eu-managers runs SELECT region, COUNT(*) FROM prod.sales.orders GROUP BY region: only EU appears and customer_email is masked. A user belonging to both support-tier2 and eu-managers sees real emails for EU rows. Support membership alone grants no row-filter bypass; a support-only user not in a permitted regional/global group sees no rows. global-finance grants all rows through this filter, but email remains masked unless the user is also exempt.
Why the alternatives are worse. A curated-view design can be secure with restricted base grants, but this requirement asks for reusable table protections across many tagged columns. Copying a masked version of the table creates stale duplicates. Revoking SELECT from analysts blocks the legitimate use of non-sensitive columns.
Common pitfalls and exam tips
- Filters and masks restrict results; they do not replace
GRANT SELECT. The user still needsUSE CATALOG,USE SCHEMAandSELECT. - A row filter function returns BOOLEAN; a column mask returns a type compatible with the masked column (prefer the exact type).
ALTER TABLE ... SET ROW FILTER ... ON (cols)andALTER COLUMN ... SET MASKattach them;DROP ROW FILTERandDROP MASKremove them.- Use
is_account_group_member()for group checks in Unity Catalog, not workspace-local group checks. - One directly attached row filter per table, one directly attached mask per column; multiple ABAC policies can apply, subject to conflict rules.
- ABAC = governed tags + policies; a policy at a catalog or schema is inherited by matching objects below it and applies to future tables too.
- For many tables with the same rule, choose ABAC; for one-off logic, a table filter or mask; dynamic views remain useful for curated/joined data without base-table access.
- Masking is not encryption and does not change stored files.
Exam tip: If a question says "hide the values of a sensitive column from most users while keeping the table readable", the answer is a column mask. If it says "each manager must see only their own region's rows", it is a row filter. If it says "the same protection for every column holding personal data across many tables, including future ones", it is an ABAC policy with governed tags.
Mini case study
Situation. A retailer's table prod.crm.customers contains customer_id, country, email and phone. Data engineers complain that marketing analysts can read emails and phone numbers. The compliance officer wants: (a) analysts see only customers of their own country; (b) email and phone hidden for everybody but the group crm-admins; (c) the phone rule to extend automatically to new tables with a phone column.
Task. Propose the design.
Solution.
- Row filter
country_filter(country)attached withALTER TABLE ... SET ROW FILTER ... ON (country); analysts are in groups per country,crm-adminssees all. - For
email, a column mask attached withALTER COLUMN email SET MASK mask_email, as this table is the only one concerned. Define a CRM-specific function that returns raw email only forcrm-admins, otherwise a constant redaction; the earlier pii-readers mask is not this requirement. - For
phone, the steward tags phone columns with a governed tag such aspii = phone, and a catalog-level ABAC column-mask policy appliesmask_phoneto tagged columns for all principals exceptcrm-admins. New tables with a tagged phone column are covered automatically. - Check that the analysts hold
USE CATALOG,USE SCHEMAandSELECT, and test with an analyst account, an admin account and an account in no group.
Key terms
- Row filter: SQL UDF returning BOOLEAN that decides which rows a user sees.
- Column mask: SQL UDF that returns the displayed value of a column.
- Dynamic view: view that changes its result depending on the reader's identity.
is_account_group_member(): function testing membership of the current user in an account-level group.- ABAC: tag/attribute-driven policies; row filters and masks restrict already permitted queries, while GRANT policies are separate.
- Governed tag: centrally controlled tag with allowed values and assignment permissions.
- Policy scope: catalog, schema or table at which an ABAC policy is defined and inherited.
- Data classification: automatic detection and tagging of sensitive data.
Summary checklist
- I can write and attach a row filter, and remove it.
- I can write and attach a column mask, and remove it.
- I can explain why filters and masks do not replace
GRANT. - I can describe governed tags, ABAC policies, their scope and inheritance.
- I can pick between a dynamic view, a table-level filter or mask, and ABAC.
- I know the main operational points: supported compute, function performance, testing.
Manage these versioned UDFs and policy definitions through the Databricks Git and Automation Bundles workflow, and validate actual group behavior alongside the gold-layer data quality checks.
Official sources
- Exam guide: https://www.databricks.com/sites/default/files/2026-10/Databricks-Certified-Data-Engineer-Associate-Exam-Guide-August-2026.pdf (opens in a new tab)
- Row filters and column masks: https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/ (opens in a new tab)
- Core concepts for attribute-based access control: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/core-concepts (opens in a new tab)
- ABAC GA announcement: https://www.databricks.com/blog/abac-row-filtering-and-column-masking-policies-governed-tags-and-data-classification-are-now (opens in a new tab)
- Manual row filters and masks: https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/manually-apply (opens in a new tab)
- ABAC policy SQL: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/policies (opens in a new tab)
- ABAC requirements: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/requirements (opens in a new tab)
- ABAC evaluation: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/policy-evaluation (opens in a new tab)
- Governed tags: https://docs.databricks.com/aws/en/admin/governed-tags/ (opens in a new tab)
- ABAC DENY scope (Beta): https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/deny-policies (opens in a new tab)
