Category index

Architect

325 articles

325 ARTICLES

FEATURED REPORT

Eclipse Dataspace Components on AWS: Data sharing fundamentals

This three-part blog series guides you through implementing Eclipse Dataspace Components (EDC) on AWS, from foundational concept to production deployment. Part 1 establishes the theoretical foundation with IDSA standards, the Dataspace Protocol (DSP), and core EDC architecture. Part 2 provides production-ready AWS deployment patterns using services like Amazon Elastic Container Service (Amazon ECS), Amazon Aurora,

BY Alejandro Esquivias Cañadas
MIN READ 1 MIN READ
EXPLORE north_east
Eclipse Dataspace Components on AWS: Data sharing fundamentals
13 hands-on demos to build on Gemini Enterprise Agent Platform
ARCHITECT

13 hands-on demos to build on Gemini Enterprise Agent Platform

Earlier this year, we introduced Gemini Enterprise Agent Platform, where you can build, scale, govern, and optimize agents. Today, we’re sharing 13 demos that walk you through what Agent Platform can do. Each one teaches a concept, a pattern, or an architecture you can put to work immediately. The best part? You don’t have to follow them step-by-step. Install Agents CLI into your favorite coding agent (Antigravity, Claude Code, Codex, whatever you use) and it instantly gets seven skills that make it an expert in ADK and Agent Platform. Describe what you want to build in plain English, and your coding agent scaffolds, evaluates, deploys, and monitors the agent for you. You’ll never have to leave your editor. Let’s dive in! Build AI agents These demos are all built on the code-first ADK. They start at the foundation and work up. 1. Start here: build your first agent with ADK. The ADK Foundation codelab is your perfect on-ramp. You set up your environment, define a basic conversational agent powered by Gemini, configure its settings, and test it through both a command-line interface and a web UI. If you’ve never touched ADK before, do this one first. 2. Build an event-driven approval agent with human-in-the-loop. The ambient expense agent codelab is the most complete “Agent Platform in action” demo in the set. You build a corporate expense agent using ADK 2.0’s graph-based workflow API. Expenses under a threshold get auto-approved in plain Python. Anything above goes through a pre-LLM security screen (PII redaction, prompt-injection defense), passes a Gemini compliance analysis, and pauses for a human-in-the-loop review before anything is finalized. You mount it behind FastAPI, trigger it from Pub/Sub events, and grade it with an LLM-as-judge eval. Keep this agent in mind – it comes back in the Scale and Govern sections. 3. Connect agents to your data with the Model Context Protocol. The MCP codelab shows you how to build reusable MCP tools that let Gemini query BigQuery, search files, and call APIs. MCP is an open protocol, so the tools you build work across different vendors and frameworks. 4. Build a dynamic frontend with Agent-to-UI (A2UI). The best user experiences are highly visual. The A2UI codelab shows you how to build an agent that renders real interface components (layouts, charts, interactive menus) that update dynamically in real time as the conversation flows. The agent literally assembles the UI the user needs, on the fly. Scale AI agents A prototype on your laptop is one thing. Handling production traffic, memory, and orchestration is what comes next. 5. Deploy a stateful data science agent to Agent Runtime (formerly known as Agent Engine). The Stateful Data Science Agent codelab walks you through building a BigQuery agent that remembers user preferences across sessions via Memory Bank, then deploying it directly to Agent Runtime. All of the underlying infrastructure, scaling, and session management are handled for you automatically. 6. Build long-running agents that pause, resume, and never lose context. Building an agent that responds to a single prompt is easy, but real enterprise workflows often take days or weeks to complete. This tutorial walks through building agents that run reliably for weeks. You’ll learn three architectural patterns: durable state machines, event-driven idle time handling, and checkpoint-and-resume with persistent sessions. The example is an onboarding coordinator agent that survives container restarts and picks up exactly where it left off. 7. Deploy an ambient expense agent to Agent Runtime with the Agents CLI. Remember the expense agent from the Build section? The Deploy to Agent Runtime codelab picks up that agent and takes it to production. You scaffold your deployment config with the Agents CLI, preview it with a dry run, then deploy it live. Cloud Trace, Cloud Logging, and BigQuery Agent Analytics wire in automatically, and the agent auto-registers in Agent Registry, so it’s discoverable across your org the moment it goes live. 8. Give your production agent a real front end. The frontend codelab is where everything comes together. You build a manager dashboard on Cloud Run, connect it to Agent Runtime through an OIDC-authenticated Pub/Sub pipeline, and give managers the ability to resume paused human-in-the-loop sessions from the browser. It ties the expense agent and the deployment together into a complete end-to-end enterprise architecture. Govern AI agents Scaling agents across an organization requires a system of built-in guardrails to manage access, track endpoints, and filter traffic. 9. Secure your agent’s lifecycle from the first commit. The Secure Agentic Coding codelab shows you how to build a shopping assistant test-first with test-driven development (TDD), wire in a custom STRIDE threat model, set up a Semgrep pre-commit hook, and configure a PreToolUse gate that blocks risky actions before execution. You deliberately plant a hardcoded API key, and the agent catches and fixes it the moment the hook fires. 10. Control agent access with Agent Gateway. The Agent Gateway codelab covers runtime governance. You deploy a multi-tool ADK agent on Agent Runtime that calls MCP servers on Cloud Run through Agent Gateway. Each agent gets a unique identity with end-to-end mTLS. Every outbound call goes through IAP authentication and IAM authorization. On top of that, Model Armor inspects all content for prompt injection and data leakage. It’s a complete, production-grade governance stack in one demo. Optimize AI agents Shipping an agent is the start. The hard part is knowing whether your next prompt tweak actually makes it better or quietly breaks ten other things. Agent Platform gives you the tools to close that loop. 11. Drive the agent quality flywheel from your coding agent. You tweaked a prompt. It looks better on three examples, but did you just break ten others? This tutorial introduces a five-stage evaluation flywheel you run directly from your coding agent: prepare data (from OTel traces, hand-crafted cases, or synthesized scenarios), run inference, grade with Google’s adaptive AutoRaters, analyze failure clusters, and execute targeted optimizations. The AutoRaters are built on the same principles Google uses to evaluate its own models and first-party agents, developed in partnership with DeepMind. Describe what you want measured in plain language. Your coding agent picks up the rest. 12. Build a cross-language multi-agent pipeline with A2A. In a large enterprise, different teams will inevitably build agents in different languages. This tutorial walks through a contract compliance pipeline where a Python-based agent extracts terms using Gemini and a Go-based agent validates them against corporate policy. The two services connect via the Agent-to-Agent (A2A) protocol and are orchestrated by ADK. You’ll learn how RemoteA2aAgent turns any A2A-compliant service into a local sub-agent with a few lines of code. 13. Scale agents across frameworks with CrewAI, LangGraph, A2A, and ADK. Most production teams don’t standardize on one agent framework. The codelab shows you how to orchestrate across all of them: an ADK control room delegates planning to a LangGraph state machine, which dispatches tasks to a CrewAI execution crew, all connected via the A2A protocol. If one step fails, the control room re-plans automatically. Get started If you want to see the full agent development lifecycle in under 10 minutes, watch this walkthrough. Otherwise, install Agents CLI, open up your coding agent, and start building today.

6 MIN READ arrow_forward
Level Up Your Column-level Security: Using IAM Data Governance Tags in BigQuery
ARCHITECT

Level Up Your Column-level Security: Using IAM Data Governance Tags in BigQuery

Many BigQuery customers rely on policy tags for protecting their sensitive information in BigQuery. Policy tags were the go-to solution for applying column-level access controls, allowing only users with the right permission to view sensitive columns like personally identifiable information (PII). It was a robust and effective system — for its time. However, data ecosystems have grown in complexity, and the tools we use to help secure them need to evolve with them. New challenges include creating and managing a taxonomy that supports multiple tags across multiple regions and locations, enabling disaster recovery, and integrating with a broad centralized governance strategy. To help you meet the needs of today’s data ecosystems, we’re excited to introduce the preview of data governance tags in BigQuery. Built on Google Cloud’s Identity and Access Manager’s (IAM) Resource Manager infrastructure, data governance tags provide a scalable, and robust method to help you manage access controls and protect your BigQuery column data. What are IAM data governance tags? Data governance tags are a special type of Resource Manager tags. You can create it by setting the purpose field to DATA_GOVERNANCE when creating a tag key in IAM, you designate it for use in BigQuery column-level security. You can create a hierarchical tree of data governance tags specifically for column-data governance purposes and apply them directly to your BigQuery columns. Why use data governance tags for column-level security? Global scope, regional enforcement: Unlike policy tags (which are regional-only), data governance tags are global. You can define a single tag key:value pair (like “data_sensitivity:high”) at the organization level and use it across any project or region in your organization. Managed disaster recovery: Security policies should persist during a failover. Data governance tags and their associated data policies are automatically replicated to secondary regions. If you need to switch regions, your security posture moves with you automatically. Hierarchical security: You can now build a tree of tags up to five levels deep. This allows for inheritance and more granular classification (such as PII > Financial > CreditCardNumber). Decoupled governance: You can tag your data to organize and classify it before you decide to enforce security. Access control only kicks in once you define a data policy for that tag, giving your team more flexibility during data onboarding. Three steps to column-level security Step 1: Create the tag key and values 1. Create data governance tag key: First you create an IAM tag key in Console, gcloud CLI, or API. The magic happens when you specify the purpose field as –purpose=DATA_GOVERNANCE for the tag key. This key change tells Google Cloud that this tag will be used for column-level security in BigQuery. code_block <ListValue: [StructValue([(‘code’, ‘# Example: Creating a Data Governance tag key named “data_class”\r\ngcloud resource-manager tags keys create data_class \\r\n –parent=projects/my-governance-project \\r\n –purpose=DATA_GOVERNANCE’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd72650460>)])]> 2. Create tag values: Once your data governance tag key has been created, you need to create specific tag values under the key that you will use to categorize/classify your column data. One of the useful features of data governance tags is the ability to build a hierarchical tree of tag values. The tag-values tree allows you to create broad categories and then drill down into specific categories based on data type. You can go up to five levels deep for granular access control. code_block <ListValue: [StructValue([(‘code’, ‘# Level 1: Create a tag value called “pii”\r\ngcloud resource-manager tags values create pii \\r\n –parent=my-governance-project/data_class\r\n\r\n\r\n# Level 2: Create a child value under “pii” for “private” data\r\ngcloud resource-manager tags values create private \\r\n –parent=my-governance-project/data_class/pii\r\n\r\n\r\n# Level 3: Create another child tag value for “email” under “private”\r\n# You can go up to 5 levels deep for granular control\r\ngcloud resource-manager tags values create email \\r\n –parent=my-governance-project/data_class/private’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd72650130>)])]> Step 2: Attach tags to your columns via JSON schema 1. Export your existing schema For existing tables, the most efficient way to manage tags is by updating the table schema using a JSON file and using API or BQ CLI because it allows you to tag multiple columns at once. code_block <ListValue: [StructValue([(‘code’, ‘# Save the current table schema to a local JSON file.\r\nbq show –schema –format=prettyjson my_project:my_dataset.my_table > schema.json’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7250b160>)])]> 2. Add the dataGovernanceTags to your JSON file Open schema.json and add the tag mapping to your sensitive columns. Note the use of the namespaced key and the short name for the value. code_block <ListValue: [StructValue([(‘code’, ‘[\r\n {\r\n “name”: “user_email”,\r\n “type”: “STRING”,\r\n “dataGovernanceTagsInfo”: {\r\n “dataGovernanceTags”: {\r\n “my-governance-project/data_class”: “email” \r\n }\r\n }\r\n },\r\n {\r\n “name”: “phone_number”,\r\n “type”: “STRING”,\r\n “dataGovernanceTagsInfo”: {\r\n “dataGovernanceTags”: {\r\n “my-governance-project/data_class”: “private”\r\n }\r\n }\r\n },\r\n {\r\n “name”: “government_id”,\r\n “type”: “STRING”,\r\n “dataGovernanceTagsInfo”: {\r\n “dataGovernanceTags”: {\r\n “my-governance-project/data_class”: “pii”\r\n }\r\n }\r\n }\r\n]’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7250b430>)])]> 3. Update the table: Apply the schema to your BigQuery table. code_block <ListValue: [StructValue([(‘code’, ‘# Overwrite the table schema with your newly tagged JSON file.\r\nbq update –project_id=my-data-project –schema=schema.json my_dataset.my_table’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7250bdc0>)])]> Alternatively you can also use SQL to bind data governance tags to BigQuery table columns. code_block <ListValue: [StructValue([(‘code’, “CREATE OR REPLACE TABLE my_dataset.my_table(\r\n user_email STRING\r\n OPTIONS (\r\n data_governance_tags = [(‘my-governance-project/data_class’, ’email’)]),\r\n );\r\nALTER TABLE my_dataset.my_table\r\nALTER COLUMN phone_number\r\n SET OPTIONS (\r\n data_governance_tags = [(‘my-governance-project/data_class’, ‘private’)]);\r\nALTER TABLE my_dataset.my_table\r\nADD COLUMN government_id\r\n STRING\r\n OPTIONS (\r\n data_governance_tags = [(‘my-governance-project/data_class’, ‘pii’)]);”), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7250b220>)])]> You can also remove a column tag by setting it to [], for example: code_block <ListValue: [StructValue([(‘code’, ‘ALTER TABLE my_dataset.my_table\r\nALTER COLUMN phone_number\r\n SET OPTIONS (\r\n data_governance_tags = []\r\n);’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7250b610>)])]> You can use information_schema COLUMNS view to see the columns tags: code_block <ListValue: [StructValue([(‘code’, “SELECT\r\n column_name,\r\n data_governance_tags[SAFE_OFFSET(0)].key AS tag_key,\r\n data_governance_tags[SAFE_OFFSET(0)].value AS tag_value,\r\nFROM my_project.my_dataset.INFORMATION_SCHEMA.COLUMNS\r\nWHERE table_name = ‘my_table’”), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7250b7c0>)])]> The result is similar to the following: code_block <ListValue: [StructValue([(‘code’, ‘+—————+———————————-+———–+\r\n| column_name | tag_key | tag_value |\r\n+—————+———————————-+———–+\r\n| user_email | my-governance-project/data_class | email |\r\n| phone_number | my-governance-project/data_class | private |\r\n| government_id | NULL | NULL |\r\n+—————+———————————-+———–+’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7271ca60>)])]> Step 3: Create data policies Finally, define a BigQuery data policy to govern access to these tagged columns. These policies explicitly reference the tag values you attached previously. Note that, while data governance tags are global, data policies are regional. To protect your data, the policy must be created in the same region where your BigQuery table is located. Once the policy is defined, access is only granted to the specified grantees; all others will be denied access to the sensitive column data. Also, keep in mind that security in BigQuery is layered. For a data policy to be effective, the users (grantees) must first possess base-level access to the table itself (typically via a role like roles/bigquery.dataViewer). Data policy then acts as a second security layer, determining whether they view the raw, sensitive column data or a masked, obfuscated version. Masking policy for ‘pii’ tagged column-data (SHA256 Masking): code_block <ListValue: [StructValue([(‘code’, ‘curl –request POST “https://bigquerydatapolicy.googleapis.com/v2/projects/myProject/locations/us-east1/dataPolicies” \\r\n –header “Authorization: Bearer $(gcloud auth print-access-token)” \\r\n –header 'Accept: application/json' \\r\n –header 'Content-Type: application/json' \\r\n –data '{\r\n “dataPolicy”: {\r\n “dataPolicyType”: “DATA_MASKING_POLICY”,\r\n “dataMaskingPolicy”: { “predefinedExpression”: “SHA256” },\r\n “grantees”: [ “principalSet://goog/group/grp-sales@corp.com” ],\r\n “dataGovernanceTag”: { “key”: “myProject/data_class”, “value”: “pii” }\r\n },\r\n “dataPolicyId”: “masking_policy_for_data_class_pii”\r\n}' \\r\n –compressed’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7271c5e0>)])]> Raw access policy for ‘pii’ tagged column-data code_block <ListValue: [StructValue([(‘code’, ‘curl –request POST “https://bigquerydatapolicy.googleapis.com/v2/projects/myProject/locations/us-east1/dataPolicies” \\r\n –header “Authorization: Bearer $(gcloud auth print-access-token)” \\r\n –header 'Accept: application/json' \\r\n –header 'Content-Type: application/json' \\r\n –data '{\r\n “dataPolicy”: {\r\n “dataPolicyType”: “RAW_DATA_ACCESS_POLICY”,\r\n “grantees”: [ “principal://goog/subject/abc@xyz.com” ],\r\n “dataGovernanceTag”: { “key”: “myProject/data_class”, “value”: “pii” }\r\n },\r\n “dataPolicyId”: “raw_access_policy_data_class_pii”\r\n}' \\r\n –compressed’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7271c460>)])]> Masking policy for “private” tagged column data (NULL Masking): code_block <ListValue: [StructValue([(‘code’, ‘curl –request POST “https://bigquerydatapolicy.googleapis.com/v2/projects/myProject/locations/us-east1/dataPolicies” \\r\n –header “Authorization: Bearer $(gcloud auth print-access-token)” \\r\n –header 'Accept: application/json' \\r\n –header 'Content-Type: application/json' \\r\n –data '{\r\n “dataPolicy”: {\r\n “dataPolicyType”: “DATA_MASKING_POLICY”,\r\n “dataMaskingPolicy”: { “predefinedExpression”: “ALWAYS_NULL” },\r\n “grantees”: [ “principal://goog/subject/abc@xyz.com” ],\r\n “dataGovernanceTag”: { “key”: “myProject/data_class”, “value”: “private” }\r\n },\r\n “dataPolicyId”: “null_policy_data_class_private”\r\n}' \\r\n –compressed’), (’language’, ‘’), (‘caption’, <wagtail.rich_text.RichText object at 0x7fdd7271c0d0>)])]> With these three steps, your column data is now protected. The next time a principal queries your BigQuery table, our authorization engine automatically evaluates their identity against your data policies. If the principal is part of the policy, they get to see the masked or raw data as per the policy; if they are not, then they will be denied access. Get started today Data governance tags are a powerful new tool to enhance your data security and governance strategy in BigQuery. We are continuously working to enhance data governance capabilities in BigQuery. Future updates include support for using SQL to create tags and tag based policies, ability to attach multiple tags to a single column, ability to define policies based on combinations of tags, and deeper integrations with services like Knowledge Catalog. You can start tagging your columns and defining fine-grained access controls at scale. To learn more, dive into the Data Governance Tags documentation.

7 MIN READ arrow_forward
Interview: Dan Cherowbrier, CTO, Formula E
ARCHITECT

Interview: Dan Cherowbrier, CTO, Formula E

For the tech chief at the electric vehicle racing organisation, innovation extends from everything digital to all the technology elements that make the growing motorsport operation a success

1 MIN READ arrow_forward
Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering
ARCHITECT

Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering

Optimizing token consumption is key to keeping AI coding assistants fast and accurate. You might not be writing every line of code any more, but now you’re responsible for directing those coding assistants to focus on getting the most out of each token. Context bloat increases latency and causes models to forget instructions or hallucinate, it also costs money and drives human attention away from the problems that actually matter. Structured habits help you maintain a fast, precise, and productive feedback loop. 1. Start with a balanced model When you are unsure, start with the default Gemini 3.5 Flash (Medium reasoning). Gauge complexity as you go. Scale up to larger models or higher reasoning if a task fails, seems to take too many hops, or needs complex design. 2. Use skills from the beginning Avoid explaining your workflow, testing rules, or environment in every prompt. Ask around, find online, or package your own reusable skills with SKILL.md files and scripts. The agent triggers them automatically, keeping prompts clean and avoiding unnecessarily searching for online docs or inspecting local code and environment. 3. Automate with scripts and CLI tools For repetitive chores like formatting many files or extracting log data, have the agent create simple local tools. Use official CLI tools for setup, linting, and testing. Run read-only commands to research the codebase before writing code, avoiding long trial-and-error loops. 4. Delegate output-heavy tasks Delegate output-heavy tasks, like deep research or separating frontend and backend work, to sub-agents. Once their work is done, you only reconcile the final results, rather than the full trajectory. 5. Divide and conquer David Rensin wrote “Elephants, Goldfish and the New Golden Age of Software Engineering” that explains how to use high-reasoning, long-context sessions (“Elephant”) to generate a detailed execution plan (the “Goldfish”). Execute that plan in a clean, low-token session. Checkpoint your progress often with commits or artifacts so you can restart from a clean state when context fills up. 6. Shift verification left Automate testing early. Run local builds and unit and functional tests before doing UI testing. Tell the agent to perform the expensive smoke-test in the browser right before handoff. Save expensive verification loops for the very end of the milestone. 7. Undo when adrift If the agent drifts and you know the fix, use the Undo button in the trajectory thread or revert your files. Do not pile corrective prompts on top of a broken state, which poisons the context. 8. Be specific with context Be specific rather than micro-managing. A clear instruction with a few spelling errors is better than a grammatically accurate broad request. Similarly, pointing the agent to the exact file, section, or error you care about (with an obvious // SHOULD BE X, NOT Y, FIX THIS annotation) instead of sending it on an open-ended search in a 10k log quest goes a long way. Whenever possible, use inline comments, so the agent knows exactly where you want the fix. 9. Iterate on rules If you keep correcting the agent’s behavior, update your global rules in AGENTS.md or edit the skill. Fix the instructions instead of prompting the agent repeatedly, so the change persists. 10. Avoid uncontrolled loops Supervisor loops that scan projects for pending work can find optimizations, but they can easily burn your entire token budget. If you run loops, set strict limits and stop conditions. High autonomy requires tighter guardrails and better evaluations. Do not let agents poll status in a loop; use event-driven wakeups. 11. Start new sessions for each new topic If you are continuing on the same topic, using the same chat can allow the agent to reuse the existing context, but if you are changing the topic, start a new chat. The agent will be able to provide better answers with fewer tokens if it only pulls in the context that it needs. Prioritize and spend wisely Tokens aren’t infinite. Behind every LLM call is a real, physical machine doing work to produce output for you. Prioritize the projects and features you care about. Token optimization is about directing the AI’s attention. By using a tiered approach you keep development fast and output sharp, while optimizing spending. We hope these 11 principles will inspire you to find the right balance between steering and automation in your AI sessions.

4 MIN READ arrow_forward
AWS Continuum to Enable Agentic Code Security for Enterprises
ARCHITECT

AWS Continuum to Enable Agentic Code Security for Enterprises

Amazon Web Services has recently introduced AWS Continuum, a new integrated security platform to automate the discovery, enforcement, and remediation of security issues across codebases, dependencies, and applications. AWS Continuum launches with four agentic capabilities, aiming at the entire vulnerability lifecycle: penetration testing, code review, threat modelling, and code vulnerabilities. By Gianmarco Nalin

1 MIN READ arrow_forward