18  Working with AI

AI-powered coding assistants can dramatically accelerate and improve your work, but they require careful and responsible use. Lab members who use AI tools must adhere to the following guidelines:

18.1 Responsibility for validation

You are fully responsible for checking and validating all AI-generated code and content. AI tools can make mistakes, generate insecure code, produce incorrect logic, or suggest approaches that are inappropriate for our specific research context. Before using any AI-generated code:

Warning

Never blindly use AI-generated code without fully understanding it. If you don’t completely understand what the AI has suggested, take the time to learn or ask a colleague for help.

18.2 Disclosure of AI use

You must clearly state whenever you have used AI tools in your work. This is essential for transparency and reproducibility. Specifically:

  • In code comments, note when AI tools were used to generate or significantly modify code
  • In commit messages, mention if AI tools assisted with the changes
  • In manuscripts and reports, acknowledge AI tool usage in the methods or acknowledgments section
  • In presentations, disclose AI assistance when relevant

Example code comment:

# The following function was generated with assistance from GitHub Copilot
# and has been reviewed and tested to ensure correctness

18.3 Attribution of sources

When using AI tools to generate content that borrows from or adapts existing sources, you must ensure proper attribution. AI tools sometimes paraphrase or adapt content from documentation, guides, or other resources without clearly indicating the original source. It is your responsibility to:

  • Ask the AI tool to identify and properly cite sources when it borrows or adapts content
  • Verify that any content the AI generates includes appropriate citations
  • Add citations yourself if the AI fails to do so
  • Follow appropriate attribution practices for the type of content (code comments, documentation, academic writing, etc.)

When instructing AI tools to create documentation or written content, explicitly request that they provide proper attribution for any borrowed or adapted material. For example: “Please quote from and paraphrase [source], with proper attribution” rather than simply asking it to summarize information on a topic.

18.4 Using AI for Journal Articles

When using AI tools to help develop journal articles and other academic writing, you must take special care to ensure transparency, maintain intellectual ownership, and avoid plagiarism. The following practices help achieve these goals.

18.4.0.1 Establish a Clear Track Record

Working with AI through GitHub creates valuable documentation of your contributions versus the AI’s. This track record can be crucial if reviewers or editors question your use of AI tools.

GitHub Pull Requests and Issues provide:

  • Attribution clarity: Each commit shows exactly who (you or @copilot) made which changes
  • Audit trail: The full conversation history shows your instructions and the AI’s responses
  • Intellectual ownership: Your prompts and guidance demonstrate that the core ideas are yours
  • Transparency: Reviewers can see that you actively supervised and validated all AI contributions

This transparency protects you if journal reviewers are skeptical about or opposed to AI use in research. You can point to the PR history to demonstrate that you maintained control and responsibility for the work.

18.4.0.2 Write Out Your Core Ideas in Prompts

Make your prompts explicit and detailed to establish that the ideas originate from you, not from the AI.

When requesting AI assistance with academic writing:

  • State your research question, hypothesis, or argument clearly
  • Outline the structure and key points you want to make
  • Specify the evidence or data you want to include
  • Describe the logic connecting your points
  • Explain the interpretation or conclusions you want to draw

Example of a good prompt:

I need help writing the discussion section for a study on social determinants of health. My core argument is that [your specific argument]. The key findings I want to discuss are: [list findings]. I want to interpret these findings as suggesting [your interpretation]. Please help me draft this section while preserving these core ideas and citing relevant literature.

This approach creates clear evidence that the intellectual content came from you, while the AI helped with expression, organization, and literature integration.

Avoid vague prompts like “write a discussion section about my results” that give the AI too much creative control and make it unclear whose ideas are being presented.

18.4.0.3 Request Explicit Source Attribution

Always instruct AI tools to identify and cite their sources to prevent unknowing plagiarism.

AI language models are trained on vast amounts of text, including published research. While they don’t have direct access to their training data during generation, they may produce text that closely resembles or paraphrases existing work without providing attribution. This creates a plagiarism risk.

Best practices:

  • Explicitly ask the AI to cite sources for any borrowed or adapted content
  • Request that the AI indicate when it is drawing from specific works
  • Verify that generated text includes proper citations
  • Add citations yourself if the AI fails to provide them
  • Cross-check AI-generated content against the cited sources to ensure accuracy

Example request:

Please help me write this section, and explicitly cite any sources you draw from or adapt. If you’re paraphrasing from specific papers, identify them clearly so I can verify the citations.

Remember that even with these precautions, you must still verify the accuracy and appropriateness of any AI-generated citations, as AI tools can sometimes generate plausible-but-incorrect references (“hallucinate” citations).

18.4.0.4 Example Workflow

The development of this documentation section demonstrates these practices. See Issue #112 and PR #145 for the complete development history, which shows:

  1. Core ideas stated clearly: The issue description outlined specific concepts (transparency through GitHub, writing explicit prompts, requesting source attribution)
  2. Detailed instructions: The guidance specified what content should be created and how it should be structured
  3. Transparent record: The development history shows what was human-directed versus AI-generated
  4. Demonstrated accountability: The pull request provides a full audit trail of all changes

When developing content for journal articles, a similar workflow creates documentation that demonstrates responsible AI use and intellectual ownership. You can point to your GitHub history to show reviewers that you directed the AI rather than simply accepting its output.

18.4.0.5 Additional Considerations

When using AI for academic writing, also remember to:

  • Disclose AI use: Follow journal policies on acknowledging AI assistance (see Section 18.2)
  • Maintain responsibility: You are accountable for all content, including AI-generated text
  • Verify accuracy: Always fact-check AI-generated claims and citations
  • Preserve your voice: Ensure the writing reflects your thinking and style, not just AI’s patterns
  • Follow ethical guidelines: Comply with your institution’s and journals’ policies on AI use

These practices help you benefit from AI assistance while maintaining the integrity, originality, and credibility of your academic work.

18.5 Coding Agents

We recommend working with AI coding agents to help you code.

18.5.1 What are AI coding agents?

AI coding agents are AI agents specialized for coding. They differ from other AI coding tools in important ways:

Compared to inline coding assistants (like traditional autocomplete), coding agents work autonomously rather than providing suggestions as you type. They can navigate entire codebases, execute commands, and complete multi-step tasks without constant human guidance.

Compared to AI chatbots (like ChatGPT or Claude), coding agents don’t just generate code snippets in conversation—they actively interact with your development environment. While chatbots require you to copy code from a chat window and manually integrate it into your project, coding agents directly read your codebase, make changes to files, run tests and build commands, and create pull requests with their proposed changes. Chatbots are conversational assistants; coding agents are autonomous development tools.

Coding agents are autonomous software programs that can:

  • Understand and execute complex tasks: Coding agents can interpret natural language instructions and break them down into actionable development tasks
  • Navigate and modify codebases: They can read, understand, and edit multiple files across a repository to implement features or fix bugs
  • Run tools and commands: Coding agents can execute build commands, run tests, use linters, and interact with development tools
  • Make decisions autonomously: They can plan their approach, make technical decisions, and adjust their strategy based on results
  • Work iteratively: Coding agents can test their changes, identify issues, and refine their solutions through multiple iterations
  • Create comprehensive solutions: They can implement complete features that span multiple files, including code, tests, and documentation

Coding agents operate in isolated environments where they can safely experiment and validate changes before proposing them. This allows them to work more independently than inline coding assistants, which require step-by-step human direction. The agent workflow typically involves analyzing requirements, planning an implementation, making changes, testing those changes, and creating a pull request with the results.

While coding agents can handle substantial development tasks, they still require human oversight and review. The human developer remains responsible for:

  • Reviewing the agent’s work
  • Ensuring the solution meets requirements
  • Verifying code quality and security
  • Making the final decision to merge changes

18.5.2 AI Agents and the Technological Singularity

The emergence of sophisticated AI agents has prompted discussions about whether we are witnessing or approaching a technological singularity. Understanding this concept helps contextualize the rapid evolution of AI tools and our responsibility in using them.

18.5.2.1 What is the technological singularity?

The technological singularity is a hypothetical future point when technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization. The concept, popularized by mathematician Vernor Vinge and futurist Ray Kurzweil, typically involves the creation of artificial superintelligence that recursively improves itself, leading to an intelligence explosion beyond human comprehension or control.

18.5.2.2 Do current AI agents represent the singularity?

No, current AI coding agents (as of early 2026) do not represent the technological singularity.

While modern AI agents demonstrate impressive capabilities, they remain fundamentally different from the singularity scenario in several critical ways:

  • Limited autonomy: Today’s AI agents operate within strict boundaries and require human oversight. They cannot recursively improve their own core architecture or develop capabilities beyond their training.

  • Narrow intelligence: AI coding agents are specialized tools designed for specific tasks. They lack general intelligence, self-awareness, or the ability to operate outside their designed domain.

  • Human dependency: These agents require human developers to: review their work, provide direction, validate correctness, and make final decisions about their outputs.

  • No recursive self-improvement: Current AI agents cannot fundamentally redesign themselves or create more advanced versions of themselves autonomously. Any improvements to AI systems still require human researchers and engineers.

  • Controlled development environment: AI coding agents work in sandboxed environments with explicit permissions and constraints. They cannot independently acquire resources, modify their own constraints, or operate without human authorization.

18.5.2.3 Why this matters for responsible AI use

Understanding that current AI agents are powerful but limited tools—not autonomous superintelligences—has important implications:

  • Maintain appropriate skepticism: AI agent outputs require the same critical review as any other tool-generated code.

  • Preserve human decision-making: The responsibility for code quality, security, and correctness remains with human developers.

  • Continue skill development: Using AI agents should enhance rather than replace human expertise.

  • Stay vigilant: While current agents don’t represent a singularity, the rapid pace of AI development requires ongoing attention to emerging capabilities and risks.

The value of AI coding agents lies in their ability to accelerate human productivity and learning, not in replacing human judgment or expertise. They are sophisticated tools that augment human capabilities while remaining under human control and oversight.

18.5.2.4 Further reading

For thoughtful perspectives on AI consciousness and intelligence, see Douglas Hofstadter’s reflections in “I Thought I Was in an AI Apocalypse. Then I Started Looking Closer.”

18.5.3 Relative Advantages of AI and Humans

AI coding agents and human coders have complementary strengths. Understanding these differences helps you decide when to delegate work to agents and when to handle tasks yourself.

18.5.3.1 Comparative Strengths: Humans vs. AI Agents

Table 18.1 summarizes the relative advantages of human coders and AI coding agents across different types of tasks:

Table 18.1: Relative advantages of humans and AI coding agents
Task Type Humans 😊 AI agents 🤖
Creative thinking 😊 Humans excel at understanding context, handling ambiguous requirements, and thinking creatively about novel problems 😞 AI agents struggle with ambiguous requirements and creative problem-solving in unfamiliar domains
Algorithmic thinking 😞 Humans make mistakes when following repetitive instructions and may introduce inconsistencies 😊 AI agents excel at executing well-defined, repetitive tasks with precision and consistency

Or, if you prefer a more visual representation:

Table 18.2: Relative advantages of humans and Agents
Humans AI Agents
Creative thinking
Algorithmic thinking

This pattern mirrors the evolution of programming itself. Just as almost no one writes machine code anymore because higher-level languages and compilers handle those details, most developers will increasingly spend less time writing low-level code. Instead, you’ll describe what the system needs to do as clearly as possible, and AI agents will handle many of the computational and coding details.

For most tasks, you won’t need to step in and manipulate code yourself. However, you’ll still need strong coding skills to:

  • Supervise and validate AI-generated code
  • Handle edge cases that agents struggle with
  • Make creative decisions about architecture and design
  • Understand when agent suggestions are incorrect or suboptimal

18.5.3.2 Future Developments: World Models

As AI technology advances, the distinction between these strengths may shift. Yann LeCun, 2019 Turing Award winner and AI researcher at Meta and NYU, advocates for developing “world models”—AI systems that understand and reason about the physical world, not just language patterns (LeCun 2022).

World models aim to give AI systems:

  • Persistent memory and reasoning: Understanding that persists across interactions
  • Physical world understanding: Reasoning about how things work in reality, not just in text
  • Better handling of ambiguity: Using world knowledge to interpret unclear requirements

As these technologies mature, AI agents may become better at tasks requiring contextual understanding and creative problem-solving. This makes it even more important to develop strong supervision and validation skills now, so you can effectively work with increasingly capable AI systems.

18.5.4 How to Work with Coding Agents

GitHub Copilot coding agents can be used in several ways to automate development tasks:

18.5.4.1 Assigning Issues to Copilot

You can assign GitHub Issues directly to @copilot just like you would assign to a human collaborator:

  1. On GitHub.com: Navigate to an issue and assign it to Copilot in the assignees section

  2. In VS Code: In the GitHub Pull Requests or Issues view, right-click an issue and select “Assign to Copilot”

  3. From Copilot Chat: Delegate tasks to Copilot directly from the chat interface in supported editors

18.5.4.2 The Agent Workflow

Once assigned an issue, the coding agent follows an autonomous workflow:

  1. Analysis: Reviews the issue description, related discussions, repository instructions, and codebase context

  2. Planning: Determines what changes are needed and creates a work plan

  3. Development: Works in an isolated GitHub Actions environment, modifies code, runs tests and linters, and validates changes

  4. Pull Request Creation: Creates a draft pull request with implemented changes, audit logs, and a summary of modifications

  5. Review and Iteration: You review the PR and can request changes; the agent will iterate based on your feedback

18.5.4.3 Example: This Document

This very section you’re reading was created through the coding agent workflow:

  1. Issue created: Issue #42 requested adding discussion about benefits and hazards of coding agents, including a Matrix film connection and best practices

  2. Agent assigned: The issue was assigned to @copilot

  3. Work completed: The agent analyzed the requirements, reviewed the repository structure, and implemented the changes across multiple files

  4. Pull request: PR #50 was created with comprehensive content about coding agents, including this “How to Work with Coding Agents” section, benefits and hazards discussion, best practices, and firewall configuration details

  5. Iteration: The PR received feedback comments requesting additional links, improved wording, and this example section—all of which the agent addressed through follow-up commits

This demonstrates the full lifecycle of working with a coding agent on a real documentation task.

18.5.4.4 Collaborating with Coding Agents

Between iterations of asking coding agents to extend a PR, human collaborators can also push changes directly to the PR branch. This allows for a collaborative workflow where both humans and agents contribute:

  • Human contributions: You can make quick fixes, add content, or refine the agent’s work by pushing commits to the same branch

  • Agent iterations: After your changes, you can ask the agent to continue working on additional requirements

Important: Try to avoid pushing changes while the coding agent is actively working. Simultaneous edits can produce conflicting diffs that:

  • Need to be manually resolved
  • May confuse both human and AI collaborators
  • Could result in lost work or merge conflicts

Best practice: Wait for the agent to complete its current iteration (indicated by the PR being updated) before pushing your own changes to the branch. Then assign new work to the agent for the next iteration.

18.5.4.5 Directly Prompting for Pull Requests

You can also prompt Copilot to create pull requests without first creating an issue:

  • Use Copilot Chat in your editor to describe the changes you want
  • The agent will analyze your request and create a pull request
  • This is useful for quick fixes or well-defined tasks

18.5.4.6 Important Safeguards

  • Human approval required: Coding agents cannot merge their own changes
  • Branch restrictions: Agents can only push to their own branches (e.g., copilot/*)
  • Full transparency: All agent actions are logged and visible in the PR

18.5.4.7 Workflow Approval Requirements

When GitHub Copilot creates or updates a pull request, it cannot automatically trigger GitHub Actions workflows. You must manually approve each workflow run by clicking the approval button in the Actions tab or on the PR.

This manual approval requirement is a security measure that prevents potentially malicious or unintended code execution. Because Copilot can modify any file in the repository—including workflow files themselves or scripts called by workflows—allowing automatic workflow execution could create security vulnerabilities.

Key points:

  • No automatic approval: There is currently no way to bypass manual workflow approval for Copilot PRs, even if you are the repository owner
  • Security reasoning: Copilot could modify workflow files (.github/workflows/*.yml) or scripts they execute, potentially injecting malicious code
  • Impact on workflow: This means you need to actively monitor and approve workflow runs as Copilot iterates on your issue, which can slow down the development cycle

Workaround considerations:

Some users have discussed using Personal Access Tokens (PATs) to allow Copilot to trigger workflows on your behalf, but this approach has security implications and should be carefully evaluated before implementation.

For more details and community discussion about this limitation, see:

For detailed instructions, see GitHub Copilot coding agent documentation.

18.5.5 Useful Prompt Formats

When working with coding agents, using clear and specific prompts helps achieve better results. Here are some useful prompt formats that you can use when requesting assistance from coding agents:

18.5.5.1 Common Task Patterns

Tidying up code:

  • “tidy up [file, function, module, whole project]”
  • Useful for improving code organization, consistency, and readability
  • Example: “tidy up the data processing module”

Addressing failing workflows:

  • “address failing workflows”
  • Helps fix continuous integration (CI) failures, build errors, or test failures
  • Example: “address failing workflows in the GitHub Actions pipeline”

Decomposing code:

  • “decompose [function, quarto-file, etc]”
  • Breaks down large or complex code into smaller, more manageable pieces
  • Example: “decompose this analysis function into separate helper functions”

Updating content:

  • “update [links, content, etc]”
  • Refreshes outdated information, fixes broken links, or modernizes code
  • Example: “update all package URLs in the documentation”

Expanding documentation:

  • “expand [a section in a document]”
  • Adds more detail, examples, or explanation to existing content
  • Example: “expand the section on data validation with practical examples”

Condensing content:

  • “condense [a section in a document]”
  • Reduces verbosity while preserving essential information
  • Example: “condense the installation instructions to be more concise”

Clarifying content:

  • “clarify [a section in a document]”
  • Improves clarity, removes ambiguity, or simplifies complex explanations
  • Example: “clarify the explanation of the analysis workflow”

18.5.5.2 Tips for Effective Prompts

  • Be specific: Include file names, function names, or specific sections when possible
  • Provide context: Explain what you want to achieve and why
  • Set boundaries: Specify what should or shouldn’t change
  • Request validation: Ask the agent to test or verify its changes when appropriate

18.5.6 Addressing Failing GitHub Actions Workflows

When GitHub Actions workflows fail, you can use Copilot to help diagnose and fix the issues. However, it’s important to use the right prompts depending on whether the problem is in your code or in the workflow configuration itself.

18.5.6.1 Scenario 1: Code Issues Found by Workflows (Most Common)

When to use: The workflow is functioning correctly, but it’s detecting problems in your code (e.g., failing tests, linting errors, build failures).

What you want: Fix the code issues without modifying the workflow files themselves.

Recommended prompts:

  • “fix the code issues found by the failing workflows”
  • “address the linting errors reported in the GitHub Actions checks”
  • “fix the test failures in the CI pipeline”
  • “resolve the build errors shown in the workflow logs”

Example: If your R package has failing tests detected by usethis::use_github_action("check-standard"), you want Copilot to fix the test failures in your R code, not modify the workflow YAML file.

Why this matters: These prompts make it clear that you want code changes, not workflow changes. This helps prevent the agent from unnecessarily modifying your carefully-configured CI/CD pipeline.

18.5.6.2 Scenario 2: Issues with Workflow Files Themselves

When to use: The workflow configuration itself has problems (e.g., syntax errors in YAML, incorrect job definitions, outdated actions).

What you want: Fix the workflow files, but with extreme caution due to security implications.

Recommended prompts:

  • “fix the syntax error in the GitHub Actions workflow file at line X”
  • “update the workflow to use the latest version of action Y”
  • “correct the job configuration in .github/workflows/check-standard.yaml”

Important considerations:

Warning

Security Warning

Workflow files have access to repository secrets and can execute arbitrary code. Before accepting any changes to workflow files:

  1. Review every line of the proposed changes
  2. Verify the changes only address the specific issue
  3. Check that no new secret access or command execution has been added
  4. Test in a safe environment if possible

See Section 18.5.8 for more details on workflow file security.

When to do it yourself: Workflow syntax errors and configuration issues are often faster to fix manually than with Copilot, especially if you’re familiar with GitHub Actions. See Section 18.5.12 for more guidance.

18.5.6.3 Scenario 3: Uncertain Which Scenario Applies

When to use: You’re not sure whether the failure is due to code issues or workflow configuration problems.

Recommended approach:

  1. First, examine the workflow logs:
    • Look at the error messages in the GitHub Actions tab
    • Identify whether the error is in your code or the workflow itself
    • Common code issues: test failures, linting errors, compilation errors
    • Common workflow issues: YAML syntax errors, missing actions, permission errors
  2. Use a diagnostic prompt:
    • “examine the failing workflow logs and identify whether the issue is in the code or the workflow configuration”
    • “diagnose the root cause of the workflow failure”
  3. Then use the appropriate scenario above: Once you understand the issue, use the specific prompts from Scenario 1 or 2.

Example workflow:

1. Prompt: "examine the failing workflow logs and identify the issue"
2. Copilot responds: "The workflow is failing because of linting errors
   in src/analysis.R"
3. Prompt: "fix the linting errors in src/analysis.R"

18.5.6.4 Additional Resources

18.5.7 Benefits and Hazards

Coding agents are powerful programs that can work autonomously. They create pull requests that propose changes to the code in our repositories, potentially including their own configuration files and our automated workflows. They can work powerfully on our behalf, but they require careful oversight and control to ensure they serve our interests and that we understand the consequences of their actions.

Coding agents offer several advantages:

  • Built-in transparency: Coding agents create a clear record of their role in your work through commit history and code suggestions

  • Context-aware suggestions: Coding agents understand your codebase and can make contextually relevant suggestions

  • Integration with version control: Using coding agents within GitHub ensures that AI-assisted changes are tracked alongside all other code changes

  • Interactive workflow: Coding agents’ interactive nature encourages you to review and modify suggestions rather than blindly accepting them

  • Accelerated development: Coding agents can help you write boilerplate code, refactor existing code, and implement common patterns more quickly

  • Learning opportunities: Coding agents can suggest approaches or techniques you may not have considered, helping you expand your coding knowledge

However, coding agents also come with significant hazards:

  • Over-reliance: Depending too heavily on coding agents can atrophy your coding skills and understanding

  • Subtle bugs: AI-generated code may contain logic errors that are not immediately obvious

  • Security vulnerabilities: Coding agents may introduce insecure patterns or fail to follow security best practices

  • Inappropriate solutions: AI may suggest solutions that work but are not optimal for your specific research context or constraints

  • Hidden biases: Coding agents may perpetuate coding patterns or approaches that reflect biases in their training data

  • False confidence: Well-formatted, professional-looking code from AI can mask underlying problems and reduce critical review

  • Workflow manipulation risks: Coding agents that modify CI/CD workflows (.github/workflows/*.yml) or setup configurations can inadvertently or maliciously compromise repository security, expose secrets, or execute harmful commands

18.5.7.1 Further reading/viewing

Three agents in suits and sunglasses from the Matrix films

Agents

18.5.8 Best Practices for Safe and Successful Use

To work with coding agents safely and successfully:

  1. Maintain active supervision: Never assume AI-generated code is correct. Review every line critically.

  2. Understand before accepting: If you don’t understand what the code does, don’t use it. Take time to learn or ask a colleague.

  3. Test thoroughly: AI-generated code must be tested as rigorously as code you write yourself. Don’t skip testing because “the AI wrote it.”

  4. Start small: Begin with small, well-defined tasks to build confidence and understanding of the agent’s capabilities and limitations.

  5. Verify logic and assumptions: Check that the AI hasn’t made incorrect assumptions about your data, requirements, or scientific context.

  6. Review for security: Explicitly check for security issues, especially when handling sensitive data or user input.

  7. Iterate and refine: Use coding agents as a starting point, not an endpoint. Refine and improve the generated code.

  8. Maintain coding practice: Regularly write code yourself to maintain and develop your skills. Don’t let the agent do everything.

WarningCritical: Exercise Extreme Caution with Workflow Files

Be especially careful when allowing coding agents to edit GitHub Actions workflows or CI/CD configurations. These files control automated processes that can:

  • Access secrets and credentials
  • Deploy code to production
  • Execute arbitrary commands in your repository

Never allow a coding agent to edit workflow files (especially .github/workflows/*.yml or copilot-setup-steps.yml) without thorough manual review. Before approving any workflow run, always check if the workflow files themselves have been modified. Malicious or erroneous changes to workflows can compromise your entire repository and its secrets.

When using coding agents, work interactively with the AI suggestions: review, modify, and test them rather than accepting them wholesale. This interactive approach helps ensure code quality and deepens your understanding of the code.

Remember: AI tools are assistants, not replacements for your expertise and judgment. The quality and correctness of your work remains your responsibility.

18.5.9 Firewall and Network Configuration

Coding agents require specific network access to function properly. If a coding agent is running behind a corporate firewall or on a restricted network, you may need to configure allowlists to enable coding agent functionality.

18.5.9.1 Built-in Agent Firewall

Coding agents run in a GitHub Actions environment with a built-in firewall that limits internet access by default. This firewall helps protect against:

  • Data exfiltration
  • Accidental leaks of sensitive information
  • Execution of malicious instructions

By default, the agent’s firewall allows access to:

  • Common OS package repositories (Debian, Ubuntu, Red Hat, etc.)
  • Popular container registries (Docker Hub, Azure Container Registry, AWS ECR, etc.)
  • Language-specific package registries (npm, PyPI, Maven, RubyGems, etc.)
  • Common certificate authorities for SSL validation

For the complete list of allowed hosts, see the Copilot allowlist reference.

18.5.9.2 Customizing Agent Firewall Settings

In your repository’s “Coding agent” settings page, you can:

  • Add custom hosts to the allowlist (for internal dependencies or additional registries)
  • Opt out of the default recommended allowlist for stricter security
  • Disable the firewall entirely (not recommended)

If a coding agent’s request is blocked by the firewall, a warning will be added to the pull request or comment, detailing the blocked address and the command that triggered it.

For more information, see Customizing or disabling the firewall for GitHub Copilot coding agent.

18.5.10 Configuring GitHub Copilot Settings

GitHub Copilot offers numerous configuration options that control how the AI assistant integrates into your development workflow. This section explains the key settings visible in your GitHub account preferences and provides guidance on which options to enable based on your use case.

18.5.10.1 Model Selection Options

GitHub Copilot provides access to multiple AI models, each with different capabilities and performance characteristics. The available models as of early 2026 include:

Anthropic Claude Models:

  • Claude Opus 4.1: Most capable model for complex reasoning and analysis
    • Pros: Excellent at understanding nuanced requirements, handling complex codebases, superior code quality
    • Cons: Slower response times, may be overkill for simple tasks, limited availability (select option required)
    • When to use: Complex refactoring, architectural decisions, thorough code reviews
  • Claude Opus 4.5: Latest version with enhanced capabilities
    • Pros: State-of-the-art performance, improved reasoning over 4.1
    • Cons: Similar trade-offs to Opus 4.1, requires selection
    • When to use: Most demanding tasks requiring cutting-edge capabilities
  • Claude Sonnet 4: Balanced model optimizing capability and speed
    • Pros: Fast responses, strong performance, good default choice
    • Cons: Slightly less capable than Opus models for very complex tasks
    • When to use: General development work, most coding tasks
  • Claude Sonnet 4.5: Enhanced version of Sonnet
    • Pros: Improved over Sonnet 4 while maintaining speed
    • Cons: Still not as powerful as Opus for extremely complex scenarios
    • When to use: Most daily development tasks
  • Claude Haiku 4.5: Fast, efficient model for simpler tasks
    • Pros: Very fast responses, cost-effective, good for quick questions
    • Cons: Less capable for complex reasoning or large codebases
    • When to use: Simple completions, quick questions, repetitive tasks

OpenAI GPT Models:

  • GPT-5.2-Codex: Specialized for code generation
    • Pros: Strong code completion, good at common patterns
    • Cons: May hallucinate package names or APIs
    • When to use: Code completion, common coding patterns
  • GPT-5: Latest general-purpose model
    • Pros: Broad knowledge, good general performance
    • Cons: Not specifically optimized for code
    • When to use: Mixed tasks involving code and documentation
  • GPT-5-Codex (various versions including Mini and Max):
    • Pros: Specialized variants for different use cases
    • Cons: Fragmented options can be confusing
    • When to use: Specific scenarios where variant optimizations matter

Google Gemini Models:

  • Gemini 2.5 Pro: High-capability model
    • Pros: Strong multimodal capabilities, good at understanding context
    • Cons: Less proven in coding scenarios than Claude or GPT
    • When to use: Tasks involving images or complex context
  • Gemini 3 Pro/Flash (Preview): Latest generation
    • Pros: Cutting-edge capabilities, flash variant offers speed
    • Cons: Preview status means less stable, limited track record
    • When to use: Experimental workflows, evaluation of new capabilities

Lab Recommendation: For most lab work, enable Claude Sonnet 4.5 as your default model. It provides excellent balance of capability and speed. Consider switching to Claude Opus 4.5 for complex architectural decisions or difficult debugging sessions. Keep Claude Haiku 4.5 enabled for quick inline completions.

18.5.10.2 Feature Settings

These settings control where and how Copilot integrates into your development environment:

Editor preview features:

  • What it does: Enables previews of experimental features in your editor
  • Pros: Access to latest capabilities before general release
  • Cons: May have bugs or unstable behavior
  • Recommendation: Enable if you’re comfortable troubleshooting issues and want cutting-edge features

Copilot Chat in GitHub.com:

  • What it does: Enables Copilot chat interface on GitHub.com
  • Pros: Quick access to Copilot without opening an editor, useful for reviewing PRs
  • Cons: Only available with paid license
  • Recommendation: Enable (included in GitHub Copilot subscription)

Copilot CLI:

  • What it does: GitHub Copilot for assistance in terminal
  • Pros: AI help for command-line operations, shell commands, and git operations
  • Cons: Requires separate installation and setup
  • Recommendation: Enable and install via gh extension install github/gh-copilot

Copilot in GitHub Desktop:

  • What it does: Enables Copilot in GitHub Desktop app
  • Pros: AI assistance in GUI git client
  • Cons: Limited compared to editor integration
  • Recommendation: Enable if you use GitHub Desktop

Copilot Chat in the IDE:

  • What it does: Enables chat interface in your code editor
  • Pros: Context-aware help, refactoring assistance, code explanation
  • Cons: Can be distracting if overused
  • Recommendation: Enable (essential feature)

Copilot Chat in GitHub Mobile:

  • What it does: Enables Copilot chat in mobile app
  • Pros: Quick access on mobile devices
  • Cons: Limited by mobile interface
  • Recommendation: Enable for convenience

Copilot can search the web:

  • What it does: Allows Copilot to search internet for up-to-date information
  • Pros: Access to current documentation, recent library changes, latest best practices
  • Cons: May introduce latency, results depend on search quality
  • Recommendation: Enable for access to current information

18.5.10.3 Advanced Settings

Dashboard Entry Point:

  • What it does: Allows instant chatting when landing on GitHub.com
  • Pros: Quick access to Copilot without navigating menus
  • Cons: None significant
  • Recommendation: Enable for convenience

Copilot code review:

  • What it does: Use Copilot to review your code and generate pull request summaries
  • Pros: Automated code review suggestions, PR summary generation, catches common issues
  • Cons: May generate false positives, shouldn’t replace human review
  • Recommendation: Enable (major productivity boost)

Automatic Copilot code review:

  • What it does: Automatically reviews all pull requests you create
  • Pros: Catches issues early without manual triggering
  • Cons: May be noisy on simple PRs, uses API quota
  • Recommendation: Disable initially; enable only after you’re comfortable with code review quality

Copilot coding agent:

  • What it does: Delegate tasks to Copilot coding agent in repositories where it is enabled
  • Pros: Autonomous multi-file edits, can execute complex refactoring, runs tests and fixes issues
  • Cons: Requires careful oversight, can make unwanted changes if instructions unclear
  • Recommendation: Enable (see Section 18.5.8 for safe usage guidelines)

Copilot Memory (Preview):

  • What it does: Remember repository context across Copilot agent interactions
  • Pros: Better context awareness, learns repository patterns and conventions
  • Cons: Preview feature governed by pre-release terms, potential privacy implications
  • Recommendation: Enable to help Copilot learn your codebase patterns

MCP servers in Copilot:

  • What it does: Connect MCP servers to Copilot in all editors and Coding Agent
  • Pros: Extend Copilot with custom tools and integrations
  • Cons: Requires MCP server setup and maintenance
  • Recommendation: Enable if you have MCP servers configured; otherwise this setting has no effect

Copilot-generated commit messages:

  • What it does: Allow Copilot to suggest commit messages when you make changes
  • Pros: Saves time, generates descriptive messages based on code changes
  • Cons: May miss important context, still requires review
  • Recommendation: Enable but always review and edit suggested messages

Copilot Spaces:

  • What it does: View and create Copilot Spaces (collaborative AI environments)
  • Pros: Share AI context with team members
  • Cons: Additional complexity for individual work
  • Recommendation: Enable for team collaboration features

Copilot Spaces Individual Access:

  • What it does: Create individually owned Copilot Spaces
  • Pros: Personal AI workspaces for complex projects
  • Cons: May fragment your workflow
  • Recommendation: Enable for flexibility

Copilot Spaces Individual Sharing:

  • What it does: Share individually owned Copilot Spaces
  • Pros: Collaborate while maintaining ownership
  • Cons: None significant
  • Recommendation: Enable for sharing capability

18.5.11 Configuring the Agent Environment

The .github/workflows/copilot-setup-steps.yml file allows you to customize the development environment in which the GitHub Copilot coding agent operates. This file preinstalls tools and dependencies so that Copilot can build, test, and lint your code more reliably.

18.5.11.1 Why Configure the Environment?

While Copilot can discover and install dependencies through trial and error, this can be slow and unreliable. Additionally, Copilot may be unable to access private dependencies. Preconfiguring the environment ensures:

  • Faster agent startup and execution
  • More reliable builds and tests
  • Access to private or authenticated dependencies
  • Consistent development environment across all agent sessions

18.5.11.2 File Location and Structure

The workflow file must be located at .github/workflows/copilot-setup-steps.yml in your repository’s default branch. It follows GitHub Actions workflow syntax but must contain a single job named copilot-setup-steps.

18.5.11.3 Basic Configuration Example

See Appendix: Copilot Setup Steps File for the configuration used in this repository (adapted for R and Quarto projects).

18.5.11.4 Using actions/checkout

The actions/checkout action is used to check out your repository code so that the workflow can access it. While Copilot will automatically check out your repository if you don’t include this step, explicitly including it is necessary when your setup steps need to access repository files.

Why explicitly include checkout?

Many dependency installation steps require access to repository files:

  • r-lib/actions/setup-renv@v2 needs renv.lock to install R package dependencies
  • r-lib/actions/setup-r-dependencies@v2 needs DESCRIPTION to install R package dependencies
  • npm ci needs package-lock.json to install Node.js dependencies
  • pip install -r requirements.txt needs the requirements file

Without an explicit checkout step, these dependency installation commands will fail because the necessary files won’t be available yet.

Basic checkout:

- name: Checkout code
  uses: actions/checkout@v4

Important: The Copilot coding agent overrides any fetch-depth value you set in the checkout step. According to GitHub’s official documentation, this override happens “to allow the agent to rollback commits upon request, while mitigating security risks.” The agent dynamically determines the appropriate fetch depth based on the pull request context.

While you cannot control the fetch depth used by Copilot, the agent still has access to sufficient git history to perform its work effectively, including comparing changes and understanding the context of your pull request.

18.5.11.5 Configurable Options

You can customize only these specific settings in the copilot-setup-steps job:

  • steps: Setup commands and actions to run
  • permissions: Access permissions (typically contents: read)
  • runs-on: Runner type (Ubuntu x64 Linux only)
  • services: Database or service containers
  • snapshot: Save environment state
  • timeout-minutes: Maximum 59 minutes

All other workflow settings are ignored by Copilot.

18.5.11.6 Common Setup Tasks

For Node.js/TypeScript projects:

- name: Set up Node.js
  uses: actions/setup-node@v4
  with:
    node-version: "20"
    cache: "npm"

- name: Install dependencies
  run: npm ci

For Python projects:

- name: Set up Python
  uses: actions/setup-python@v5
  with:
    python-version: "3.11"

- name: Install dependencies
  run: pip install -r requirements.txt

For R projects:

- name: Set up R
  uses: r-lib/actions/setup-r@v2
  with:
    r-version: 'release'

- name: Install R dependencies
  uses: r-lib/actions/setup-renv@v2

18.5.11.7 Environment Variables and Secrets

To set environment variables for Copilot:

  1. Navigate to your repository’s Settings
  2. Go to Environments
  3. Select or create the copilot environment
  4. Add environment variables or secrets as needed

Use secrets for sensitive values like API keys or passwords.

18.5.11.8 Testing Your Configuration

The workflow runs automatically when you modify copilot-setup-steps.yml, allowing you to validate changes in pull requests. You can also manually trigger the workflow from the repository’s Actions tab.

Setup logs appear in the agent session logs when Copilot starts working. If a step fails, Copilot will skip remaining steps and begin working with the current environment state.

18.5.11.9 Advanced Configuration

Larger runners: For projects requiring more resources, you can use larger GitHub-hosted runners:

jobs:
  copilot-setup-steps:
    runs-on: ubuntu-4-core

Self-hosted runners (ARC): For access to internal resources or private registries, use Actions Runner Controller (ARC) self-hosted runners:

jobs:
  copilot-setup-steps:
    runs-on: arc-scale-set-name

Note: When using self-hosted runners, you must disable Copilot’s integrated firewall in repository settings and configure appropriate network security controls.

Git Large File Storage (LFS): If your repository uses Git LFS:

- uses: actions/checkout@v4
  with:
    lfs: true

18.5.11.10 Further Reading

For complete details, see Customizing the development environment for GitHub Copilot coding agent.

18.5.12 When to use a coding agent

Coding agent sessions are currently1 considered “premium requests”, which are limited resources; see https://github.com/features/copilot/plans for details. So, use coding agents sparingly. Use them for complex changes that would be difficult or time-consuming for you to complete by hand. Coding agents also take time to get configured for work, every time you make a request. See https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment#preinstalling-tools-or-dependencies-in-copilots-environment for ways to reduce that startup time, but it will never be 0. If you can complete the task faster than the coding agent can, you should probably do it yourself. For example, when you have errors in the spell-check or lint workflows, you can often fix them faster than Copilot can. Similarly, when reviewing Copilot’s PRs, you can often make direct changes to the branch faster than you could write clear review comments and get Copilot to address them.

Also, the less we practice, the weaker our skills get, and the harder it is for us to supervise the agents and make sure they are actually doing what we want them to do, the way we want them to do it. You should exercise your own coding skills regularly, just like you would for any other skill you want to maintain.

18.5.13 Editing with .docx files

GitHub Copilot coding agents can read Microsoft Word (.docx) files, including tracked changes and comments. This enables a hybrid editing workflow where:

  1. Lab members can export Quarto content to Word format for review
  2. Reviewers can make edits, add tracked changes, and insert comments in Word
  3. Coding agents can read the .docx file and translate the edits back to Quarto format

When using this workflow, make sure to explicitly instruct the coding agent to:

  • Examine and apply all tracked changes in the .docx file
  • Read and address all comments in the .docx file
  • Translate edits from Word formatting to appropriate Quarto/markdown syntax

This approach makes it easier for collaborators who are more comfortable with Word to contribute to the lab manual while maintaining the source files in Quarto format.

18.5.13.1 Known Issue: “Document 1” Warning in Word

When opening DOCX files generated by Quarto (including this lab manual), Microsoft Word may display a warning message and open the file with the title “Document 1” instead of the actual filename. Word may also require you to save the file before you can add comments or track changes.

This is a known limitation with how Quarto generates DOCX files. The issue is being tracked in the Quarto project:

Workaround: If you are the author generating the DOCX file from Quarto, follow these steps before sharing with collaborators:

  1. Open the generated DOCX file in Microsoft Word
  2. Immediately save the file (File → Save, or Ctrl+S/Cmd+S)
  3. Close and re-open the file to verify it no longer shows “Document 1”
  4. Share this saved version with collaborators

This one-time step ensures that when collaborators open the file, they won’t see the “Document 1” warning and can immediately add comments and track changes without issues.

18.5.14 Copilot Instructions for this Repository

The .github/copilot-instructions.md file in this repository contains specific instructions and guidelines for GitHub Copilot coding agents when working with the lab manual. This file helps ensure that AI-generated contributions follow the lab’s formatting standards, coding conventions, and documentation practices.

The copilot instructions file specifies:

  • Markdown and Quarto formatting rules (e.g., blank lines before lists, line breaks in prose)
  • R code style guidelines (e.g., using native pipe |>, following tidyverse style)
  • File organization patterns (e.g., using Quarto includes for modular content)
  • How to work with DOCX files for hybrid editing workflows
  • Repository-specific best practices

By having these instructions in .github/copilot-instructions.md, we ensure that coding agents produce consistent, high-quality contributions that align with the lab’s established practices. This reduces the review burden and helps maintain consistency across all contributions to the lab manual, whether made by humans or AI assistants.

See Appendix: Copilot Instructions File for the complete file.

18.5.15 Using Copilot Review Before Human Review

Before requesting review from other humans, always have Copilot review your pull request first—even if Copilot created the PR itself. AI review provides fast, thorough feedback that helps catch issues before involving human reviewers, saving everyone time and improving code quality.

Why review with Copilot first:

  • AI has more bandwidth: Copilot can review code immediately without competing priorities
  • Catch common issues early: Copilot excels at identifying bugs, logic errors, security vulnerabilities, and style inconsistencies
  • Improve human review quality: When humans review cleaner code, they can focus on higher-level concerns like design and architecture rather than basic issues
  • Learn from feedback: Even experienced developers benefit from Copilot’s perspective on best practices and potential improvements
  • Growing capabilities: AI review capabilities continue to improve over time, making this investment increasingly valuable

Copilot review workflow:

  1. Assign Copilot as a reviewer: On your pull request page, assign Copilot to review the PR the same way you would assign any other reviewer. Click “Reviewers” in the right sidebar and select Copilot from the list.

  2. Review Copilot’s comments: Once Copilot completes its review, carefully examine each comment. For each comment, decide whether you agree with the suggestion:

    • If the comment is correct: Address it by making code changes yourself or ask Copilot to apply the fix using GitHub’s suggestion features
    • If the comment is incorrect or not applicable: Dismiss the comment with an explanation for why it doesn’t apply
    • If you’re uncertain: Seek a second opinion from a human reviewer or do additional research

  3. Request another Copilot review: After addressing or dismissing all comments, request another review from Copilot. This creates an iterative improvement process.

  4. Iterate until satisfied: Repeat the review-and-address cycle until Copilot stops providing valuable suggestions. This typically takes 1-3 iterations depending on the complexity of the changes.

  5. Request human review: Only after you’ve addressed Copilot’s feedback should you request review from human team members. At this point, the code should be in better shape, allowing human reviewers to focus on higher-level concerns.

Important considerations:

  • Copilot isn’t perfect: AI review can produce false positives or miss important issues. Always apply your own judgment when evaluating Copilot’s suggestions.
  • Don’t blindly accept all suggestions: Some of Copilot’s recommendations may not fit your specific context or requirements. It’s perfectly appropriate to dismiss comments that don’t apply.
  • Human review remains essential: Copilot review supplements but does not replace human code review. Humans bring domain knowledge, understanding of business requirements, and judgment about trade-offs that AI cannot replicate.
  • Document dismissals: When dismissing Copilot comments, briefly explain why. This helps human reviewers understand your reasoning and can serve as documentation for future reference.

For pull request authors:

Even if you’re highly experienced, treating Copilot review as a required pre-review step helps maintain code quality and makes the best use of everyone’s time. The few minutes spent on Copilot review often save hours of back-and-forth with human reviewers.

For human reviewers:

When you receive a PR for review, check whether the author has completed the Copilot review process. If Copilot hasn’t reviewed the PR yet, consider asking the author to complete that step first before you invest time in review. This ensures you’re reviewing code that has already been through initial automated quality checks.

18.5.16 Reviewing a Copilot PR You Didn’t Create

When reviewing a pull request where someone else prompted Copilot to make changes, follow these guidelines to avoid confusion and ensure smooth collaboration:

Understanding PR roles:

Different people may play different roles in the PR lifecycle:

  • Issue creator: Reports a bug, suggests a feature, or requests other improvements. In projects with external users, issue creators often cannot assign issues to developers and are therefore distinct from the PR prompter
  • PR prompter: Assigns a developer (human or AI) to start working on a PR, often by assigning an issue to Copilot. The PR prompter is sometimes the same person as the issue creator, but often is a project maintainer who reviews and triages external issue reports
  • PR author(s): Makes commits to the PR branch (when Copilot creates commits, both Copilot and the prompter are co-authors)
  • PR manager: Supervises and guides the PR authors, assigns reviewers, and controls the PR workflow. Typically the PR prompter becomes the PR manager, but can hand off this role to someone else
  • PR reviewer(s): Reviews the PR and provides feedback. The PR manager often also serves as the lead reviewer
  • PR assignee(s): Listed in the “Assignees” field on GitHub to indicate who is responsible for the PR. Use this field to clarify current ownership and track who should be working on addressing feedback

The scenario:

  • One team member (the “PR prompter”) assigned Copilot to work on an issue or explicitly prompted Copilot to start working
  • The prompter may or may not be the same person who originally created the issue
    • In projects with a user base, users often submit issues (bug reports, feature requests)
    • A project maintainer then steps in, adds their perspective, and assigns the issue to Copilot
    • In this case, the maintainer who assigned Copilot is the prompter, not the original issue creator
  • Copilot created the PR with the prompter as co-author
  • The prompter (now acting as PR manager) requested your review
  • Copilot may have also automatically reviewed the PR

As a non-manager reviewer, your role is to provide feedback, not to directly initiate more work by Copilot. The PR manager should remain in control of when and how Copilot makes additional changes.

Recommended review workflow:

  1. Use “Comment” or “Request changes” based on the severity of issues:
    • Use “Comment” for suggestions, questions, or minor issues that don’t block merging
    • Use “Request changes” for significant issues that must be addressed before merging
    • Both options allow you to provide feedback without directly triggering Copilot
  2. Don’t ask Copilot to make changes directly:
    • Avoid using features that would trigger Copilot to start working immediately
    • Let the PR manager decide whether to ask Copilot to address your comments or make changes themselves
  3. Write clear, actionable comments:
    • Explain what needs to change and why
    • Suggest specific solutions when appropriate
    • The PR manager will decide how to address your feedback

For PR managers:

After receiving reviews from other team members:

  1. Review all comments carefully:
    • Decide which comments you agree with
    • Dismiss or respond to comments you don’t entirely agree with
    • This ensures Copilot only addresses feedback you’ve validated
  2. Choose how to address valid feedback:
    • Option A: Make the changes yourself (faster for simple fixes)
    • Option B: Ask Copilot to address the feedback (better for complex changes)
    • Option C: Add your own review summarizing which comments Copilot should address, then ask Copilot to respond to the open comment threads
  3. Maintain clear communication:
    • Let reviewers know how you plan to address their feedback
    • Mark conversations as resolved after addressing them
    • Request re-review from humans after Copilot makes significant changes
    • Update the PR’s “Assignees” field to reflect who is currently responsible for the PR

Transferring the PR manager role:

The original PR manager can hand over a PR to another person, who then becomes the new PR manager with control over Copilot’s work on that PR. This might be useful when:

  • The original PR manager is unavailable or on leave
  • Someone with different expertise needs to guide the remaining work
  • Responsibilities are being redistributed within the team

To transfer the PR manager role:

  1. The original PR manager should clearly communicate the handover to all reviewers
  2. The new PR manager should review the PR’s history and any open feedback
  3. The new PR manager should take over responding to Copilot and managing future iterations
  4. The team should update the PR’s “Assignees” field and comments or description to reflect the current PR manager

This workflow ensures the PR manager maintains control over the development process while benefiting from collaborative human review and Copilot’s implementation capabilities.

  1. 2026-01-10↩︎