Skip to content

Unity Catalog Row Filters, Column Masks and ABAC Policies

Step 29 of 30

Unity Catalog Row Filters, Column Masks and ABAC Policies

Introduction

This lesson completes the Governance and Security section of the official exam outline (about 15% of the scored questions). The Unity Catalog tables and privileges lesson established object-level grants. This lesson applies Unity Catalog row filters, column masks and ABAC for the Databricks Data Engineer Associate exam. At the level of whole objects, a principal can or cannot read a table. Real organizations need finer control. A regional sales manager should see only the rows of their region; a support agent should see the last four digits of a card number but not the whole value; an analyst should see the data but never the email addresses.

Unity Catalog offers three tools for this fine-grained control: row filters, column masks and, at scale, attribute-based access control (ABAC) policies driven by governed tags. Row filters and column masks are defined per table. ABAC applies the same logic automatically to every table that carries the right tags, across a whole catalog or schema. Databricks announced ABAC row filtering and column masking policies, governed tags and data classification as generally available in 2026; its documentation recommends ABAC for consistent rules across many tables, and per-table filters and masks when you need logic specific to one table or have not adopted ABAC yet.

Learning objectives

  • Explain the difference between object-level privileges, row filters, column masks and ABAC policies.
  • Write a row filter function and attach it to a table.
  • Write a column mask function and attach it to a column.
  • Use is_account_group_member() to implement identity-based logic.
  • Explain ABAC: governed tags, policy scope, principals, conditions and the policy functions.
  • Choose between a dynamic view, per-table filters/masks and ABAC.
  • Name the limits and operational points to remember (privileges, performance, and what consumers see).

1. Why object-level privileges are not enough

Imagine the table prod.sales.orders, with columns order_id, region, customer_email and amount.

Requirement Object-level GRANT SELECT What is needed
EU managers see only EU rows Cannot express it Row filter
Everybody sees the table but the email is hidden except for support Cannot express it Column mask
All columns tagged as personal data are masked in 400 tables Would need 400 separate definitions ABAC policy with governed tags

Before these features, engineers created dynamic views per table, with a CASE expression or a WHERE clause using is_account_group_member(). A dynamic view remains useful for curated or joined datasets; finding a base table does not grant access to it. Restrict base-table SELECT when a view is the intended security boundary. A view can combine multiple tables rather than requiring exactly one view per table. Row filters and masks are attached to the table itself, so supported Unity Catalog query paths enforce them. This is not protection against independent cloud-IAM access to raw files, nor a guarantee that every external API or older compute mode supports the table.

2. Row filters

A row filter is a SQL user-defined function (UDF) that returns a BOOLEAN. It is attached to a table and receives the values of one or more columns of each row. Unity Catalog evaluates it for every query on the table; rows for which it returns TRUE are returned, and rows for which it returns FALSE (or NULL) are hidden. The reader does not see the filter in the query: the rows are simply absent.

SQL
-- 1. The function
CREATE OR REPLACE FUNCTION prod.security.region_filter(region STRING)
RETURNS BOOLEAN
RETURNS BOOLEAN
RETURNS BOOLEAN
RETURN is_account_group_member('global-managers')
    OR (is_account_group_member('eu-managers') AND region = 'EU')
    OR (is_account_group_member('us-managers') AND region = 'US');

-- 2. Attach it to the table, naming the column(s) passed to the function
ALTER TABLE prod.sales.orders
SET ROW FILTER prod.security.region_filter ON (region);

-- Optional cleanup exercise only; this removes the attached protection.
-- Run separately from installation, after reviewing the access requirement.
ALTER TABLE prod.sales.orders DROP ROW FILTER;

Points to remember:

  • The function is a normal Unity Catalog object, so it can be stored in a dedicated security schema with its own, tightly controlled privileges.
  • is_account_group_member('group') returns true when the current user belongs to the account-level group; it is the preferred check in Unity Catalog. current_user() returns the user's identity and is useful for a mapping table (for example user_regions), but a mapping table must itself be protected.
  • A table can have one row filter; the function may combine several conditions.
  • The filter can also be given other columns, or a constant argument, with types compatible with the SQL UDF parameters. Keep it simple and efficient; identity checks intentionally depend on the query identity. Do not infer that arbitrary non-deterministic logic, complex subqueries or external calls are supported in every read/write operation.
  • Users need access to the table (SELECT and the USE privileges) as usual; the row filter then trims what they receive. Attaching/removing a filter on an existing table requires table ownership or both MANAGE and SELECT, plus EXECUTE on the function and parent usage privileges. Creating the function also requires the relevant schema rights.

Concrete example. A bank stores loans(loan_id, branch_id, amount, ...). Branch managers must see only their branch. The security team creates branch_filter(branch_id) that compares branch_id with a lookup in a protected mapping table security.user_branches using current_user(). A manager runs SELECT COUNT(*) FROM loans and gets only the loans of their branch; an auditor in the group audit gets everything.

3. Column masks

A column mask is a SQL UDF attached to a column. It takes the column's value (and optionally other columns of the row) and returns the value to display. The return type must match or be castable to the column's type; prefer an exact type match, so that queries and tools do not break.

SQL
CREATE OR REPLACE FUNCTION prod.security.mask_email(email STRING)
RETURNS STRING
RETURN CASE
  WHEN is_account_group_member('pii-readers') THEN email
  ELSE '[REDACTED]'
END;

ALTER TABLE prod.sales.orders
ALTER COLUMN customer_email SET MASK prod.security.mask_email;

-- Optional cleanup: removes the column protection; not an installation step.
ALTER TABLE prod.sales.orders
ALTER COLUMN customer_email DROP MASK;

Members of pii-readers see maria@example.com; other permitted readers see [REDACTED], without exposing the domain or malformed original strings. These snippets assume the named schema, groups and STRING columns exist; test them on an isolated practice table.

A mask can use additional columns of the same row through USING COLUMNS:

SQL
CREATE OR REPLACE FUNCTION prod.security.mask_card(card STRING, country STRING)
RETURNS STRING
RETURN CASE
  WHEN is_account_group_member('payment-readers') AND country IN ('FR', 'DE')
    THEN card
  ELSE concat('****', right(card, 4))
END;

ALTER TABLE prod.sales.orders
ALTER COLUMN card_number SET MASK prod.security.mask_card USING COLUMNS (country);

Properties:

Aspect Row filter Column mask
Attached to A table A column
Function returns BOOLEAN Same type as the column
Effect Hides rows Replaces values
Number per object One per table One per column
Removing ALTER TABLE ... DROP ROW FILTER ALTER COLUMN ... DROP MASK

Masks are not encryption. They control what a query returns. Seeing raw values depends on the actual mask logic and applicable policy exemptions, not simply on being an owner or administrator. Principals able to change a table-level mask or its UDF can change the protection; separate those duties. Stored files are unchanged, so also restrict direct cloud access.

4. Dynamic views versus filters and masks

Criterion Dynamic view Row filter / column mask ABAC policy
Protection point A view, with base access restricted by its own grants The table itself The table itself, by tags
Reuse A curated view can join/reshape several tables One function reused by many tables, attached manually One policy for many tables automatically
Effort for 400 tables Very high High (attach to each) Low (tag once, policy applies)
Typical use today Curated, transformed or joined access without base-table grants Table-specific logic Consistent enterprise rules

5. Attribute-based access control (ABAC)

ABAC row-filter and column-mask policies restrict permitted queries using attributes of the data (tags) and, in policies, of the principals (their groups), in addition to the object privileges that allow a query. Separate ABAC GRANT policies can grant privileges; the filter/mask policies taught here do not. In Unity Catalog the building blocks are:

  • Governed tags: account-level tags whose name, allowed values and the permission to assign them are controlled centrally. Unlike free tags, a governed tag cannot be created or applied by just anyone; a data steward decides who may assign pii = email, for example.
  • Policies: rules that say "for the objects matching this tag condition, apply this row filter or column mask function, to these principals, except those principals".
  • Policy functions: the same kind of SQL UDFs as in sections 2 and 3, now referenced by a policy.

A policy is defined on a scope: a catalog, a schema or a table. It is inherited downward, so a policy at catalog level applies to all current and future matching tables below it. The condition uses the tags of the tables and columns, and the principals can be users, service principals or groups, with an EXCEPT list for those not concerned.

The following policy masks matching email columns for everyone except the privacy team using the documented SQL syntax. This example targets STRING columns directly tagged with the governed pii=email value. The mask UDF always redacts; exemptions belong in the policy rather than hidden group checks in the UDF:

SQL
CREATE OR REPLACE FUNCTION prod.security.redact_email(email STRING)
RETURNS STRING
RETURN CASE WHEN email IS NULL THEN NULL ELSE '[REDACTED]' END;

CREATE POLICY mask_emails
ON CATALOG prod
COLUMN MASK prod.security.redact_email
TO `account users`
EXCEPT `pii-readers`
FOR TABLES
MATCH COLUMNS has_tag_value('pii', 'email') AS email_col
ON COLUMN email_col;

A row filter policy follows the same pattern, with ROW FILTER and a condition on the table, for example tables tagged domain = sales that have a column tagged region.

Operational facts:

  • Creating a policy requires the right to manage it on the securable (the MANAGE privilege on the catalog, schema or table, or ownership) and EXECUTE on the function used.
  • Tagging the data is the key step: a column without the tag is not affected. Governance teams can use data classification to detect and tag sensitive columns. Native classifier tags such as class.email_address are not automatically the custom pii=email taxonomy used here: configure the matching policy/tagging process explicitly. Column tags do not inherit from a table/catalog; matching column tags must be applied directly. Tag assignment requires ASSIGN on the governed tag and APPLY TAG on the object.
  • Policies are evaluated at query time through supported Unity Catalog query interfaces.
  • Keep current scope boundaries clear: metastore-level policies and ABAC on views have separate Beta requirements, and ABAC DENY policies (Beta) currently target MANAGE ACCESS CONTROL. Do not treat a Beta DENY policy as a general DENY SELECT statement. Advanced identity-attribute functions have their own documented contexts and limitations; they are not needed for the group-based examples here.

Concrete example. A pharmaceutical group has 1,200 tables across 30 schemas. Compliance requires that every column holding a patient identifier is masked for everybody except the clinical-data team. Without ABAC, an engineer would attach a mask to each column, then forget new tables. With ABAC, the data steward tags the identifier columns with pii = patient_id (through a tagging/classification workflow configured for that taxonomy), one catalog-level policy applies mask_patient_id to every tagged column, now and in the future, and the clinical-data group is the exception.

6. Operational points

  • Compute: table-level filters/masks support SQL warehouses, standard DBR 12.2 LTS+ and dedicated DBR 15.4 LTS+ with required serverless filtering. ABAC policies require serverless or standard DBR 16.4+, or dedicated DBR 16.4+ with fine-grained filtering enabled. Dedicated writes and specific DML patterns have additional version limits; verify them rather than treating all modes as equivalent.
  • Performance: the function runs on the query path. Keep it simple: no heavy subqueries or external calls. A mapping-table lookup should be small.
  • Downstream objects: for a supported view over an ABAC-protected table, base privileges are checked for the view owner while policies are evaluated for the querying user. For materialized/streaming refreshes, verify the pipeline owner/run-as context. A persisted downstream result does not automatically establish the desired permissions or identity-dependent protection on the target. Secure that object explicitly and test through the actual consumer interface.
  • Testing: test with accounts in each group, and check the result of SELECT *, COUNT(*), joins and aggregations, because a mask applied before an aggregation changes the aggregate.
  • Auditing: inspect query access and tag/policy changes through supported audit/system-table views. Test overlapping masks/filters: conflicting applicable functions can fail a query rather than selecting the last policy.

Worked example: protecting customer data in a shared orders table

Scenario. prod.sales.orders has order_id, region, customer_email, card_last4 and amount. Requirements:

  1. Regional managers see only their region's rows; the group global-finance sees everything.
  2. Everybody except support-tier2 sees customer_email masked.
  3. The same email rule must apply to 80 other tables in the catalog prod.

Step 1: Row filter on this table. Table-specific logic fits a per-table row filter.

SQL
CREATE OR REPLACE FUNCTION prod.security.region_filter(region STRING)
RETURNS BOOLEAN
RETURN is_account_group_member('global-finance')
    OR (is_account_group_member('eu-managers') AND region = 'EU')
    OR (is_account_group_member('us-managers') AND region = 'US');

ALTER TABLE prod.sales.orders SET ROW FILTER prod.security.region_filter ON (region);

Step 2: Email masking for 80 tables. Use the unconditional redact_email UDF above, have the steward directly tag every email column with pii=email, and create the catalog-level policy with support-tier2 in its EXCEPT list instead of pii-readers. Do not reuse the earlier conditional mask_email as the policy function: it would also unmask pii-readers and violate this scenario. Do not attach that separate per-table email mask to the same column in this scenario.

Step 3: Grants. Make sure the groups hold USE CATALOG, USE SCHEMA and SELECT, because filters and masks restrict what is returned and do not grant access.

Step 4: Test. A user in eu-managers runs SELECT region, COUNT(*) FROM prod.sales.orders GROUP BY region: only EU appears and customer_email is masked. A user belonging to both support-tier2 and eu-managers sees real emails for EU rows. Support membership alone grants no row-filter bypass; a support-only user not in a permitted regional/global group sees no rows. global-finance grants all rows through this filter, but email remains masked unless the user is also exempt.

Why the alternatives are worse. A curated-view design can be secure with restricted base grants, but this requirement asks for reusable table protections across many tagged columns. Copying a masked version of the table creates stale duplicates. Revoking SELECT from analysts blocks the legitimate use of non-sensitive columns.

Common pitfalls and exam tips

  • Filters and masks restrict results; they do not replace GRANT SELECT. The user still needs USE CATALOG, USE SCHEMA and SELECT.
  • A row filter function returns BOOLEAN; a column mask returns a type compatible with the masked column (prefer the exact type).
  • ALTER TABLE ... SET ROW FILTER ... ON (cols) and ALTER COLUMN ... SET MASK attach them; DROP ROW FILTER and DROP MASK remove them.
  • Use is_account_group_member() for group checks in Unity Catalog, not workspace-local group checks.
  • One directly attached row filter per table, one directly attached mask per column; multiple ABAC policies can apply, subject to conflict rules.
  • ABAC = governed tags + policies; a policy at a catalog or schema is inherited by matching objects below it and applies to future tables too.
  • For many tables with the same rule, choose ABAC; for one-off logic, a table filter or mask; dynamic views remain useful for curated/joined data without base-table access.
  • Masking is not encryption and does not change stored files.

Exam tip: If a question says "hide the values of a sensitive column from most users while keeping the table readable", the answer is a column mask. If it says "each manager must see only their own region's rows", it is a row filter. If it says "the same protection for every column holding personal data across many tables, including future ones", it is an ABAC policy with governed tags.

Mini case study

Situation. A retailer's table prod.crm.customers contains customer_id, country, email and phone. Data engineers complain that marketing analysts can read emails and phone numbers. The compliance officer wants: (a) analysts see only customers of their own country; (b) email and phone hidden for everybody but the group crm-admins; (c) the phone rule to extend automatically to new tables with a phone column.

Task. Propose the design.

Solution.

  1. Row filter country_filter(country) attached with ALTER TABLE ... SET ROW FILTER ... ON (country); analysts are in groups per country, crm-admins sees all.
  2. For email, a column mask attached with ALTER COLUMN email SET MASK mask_email, as this table is the only one concerned. Define a CRM-specific function that returns raw email only for crm-admins, otherwise a constant redaction; the earlier pii-readers mask is not this requirement.
  3. For phone, the steward tags phone columns with a governed tag such as pii = phone, and a catalog-level ABAC column-mask policy applies mask_phone to tagged columns for all principals except crm-admins. New tables with a tagged phone column are covered automatically.
  4. Check that the analysts hold USE CATALOG, USE SCHEMA and SELECT, and test with an analyst account, an admin account and an account in no group.

Key terms

  • Row filter: SQL UDF returning BOOLEAN that decides which rows a user sees.
  • Column mask: SQL UDF that returns the displayed value of a column.
  • Dynamic view: view that changes its result depending on the reader's identity.
  • is_account_group_member(): function testing membership of the current user in an account-level group.
  • ABAC: tag/attribute-driven policies; row filters and masks restrict already permitted queries, while GRANT policies are separate.
  • Governed tag: centrally controlled tag with allowed values and assignment permissions.
  • Policy scope: catalog, schema or table at which an ABAC policy is defined and inherited.
  • Data classification: automatic detection and tagging of sensitive data.

Summary checklist

  • I can write and attach a row filter, and remove it.
  • I can write and attach a column mask, and remove it.
  • I can explain why filters and masks do not replace GRANT.
  • I can describe governed tags, ABAC policies, their scope and inheritance.
  • I can pick between a dynamic view, a table-level filter or mask, and ABAC.
  • I know the main operational points: supported compute, function performance, testing.

Manage these versioned UDFs and policy definitions through the Databricks Git and Automation Bundles workflow, and validate actual group behavior alongside the gold-layer data quality checks.

Official sources