Testing Methodology

📖 This is a condensed reference. For the full guide, see the official Frappe docs: Unit Testing Guide · Running Tests

---

Overview

Frappe's test runner is built on Python's unittest module and integrates directly with Bench. Key rules:

Rule	Detail
File naming	Must start with test_ and be a .py file
Test site	Must run on a site starting with test_ (prevents accidental data loss)
Auto stubs	Test stubs are auto-generated when a new DocType is created
Auto fixtures	Test runner builds records for all linked (FK) DocTypes automatically
Run command	bench run-tests

---

Writing DocType Tests

Test files live alongside the DocType: test_[doctype].py. Use FrappeTestCase (not bare unittest.TestCase) — it resets frappe.local.flags and rolls back the DB transaction after each test.

import frappe
from frappe.tests.utils import FrappeTestCase

def create_events():
    if frappe.flags.test_events_created:
        return
    frappe.set_user("Administrator")
    frappe.get_doc({"doctype": "Event", "subject": "_Test Event 1",
                    "starts_on": "2024-01-01", "event_type": "Public"}).insert()
    frappe.get_doc({"doctype": "Event", "subject": "_Test Event 2",
                    "starts_on": "2024-01-01", "event_type": "Private"}).insert()
    frappe.flags.test_events_created = True

class TestEvent(FrappeTestCase):
    def setUp(self):
        create_events()

    def tearDown(self):
        frappe.set_user("Administrator")

    def test_allowed_public(self):
        frappe.set_user("test1@example.com")
        doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject": "_Test Event 1"}))
        self.assertTrue(frappe.has_permission("Event", doc=doc))

    def test_not_allowed_private(self):
        frappe.set_user("test1@example.com")
        doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject": "_Test Event 2"}))
        self.assertFalse(frappe.has_permission("Event", doc=doc))

Why FrappeTestCase?

• Resets frappe.local flags between tests
• Wraps each test in a DB transaction → rolled back automatically — no leftover data
• Always call super().setUpClass() when overriding setUpClass

---

Running Tests

# All tests
bench run-tests

# By app
bench run-tests --app erpnext

# By DocType
bench --site test.local run-tests --doctype "Sales Order"

# Specific test method
bench run-tests --doctype User --test test_get_value

# By module (dotted path)
bench run-tests --module erpnext.support.doctype.issue.test_issue

# With Python profiler
bench run-tests --doctype "Activity Cost" --profile

# Skip fixture creation (faster during dev)
bench --site test.local run-tests --doctype "Student Group" --skip-test-records --skip-before-tests

# Verbose output
bench --verbose run-tests

# JUnit XML output (for CI)
bench run-tests --junit-xml-output=/reports/junit_test.xml

---

Parallel Execution

# Static split — divide test files evenly across N instances
bench --site test.local run-parallel-tests --build-id 1 --total-builds 2

# Dynamic split — uses Frappe Test Orchestrator for time-balanced distribution
bench --site test.local run-parallel-tests --use-orchestrator

The orchestrator assigns the next test file to whichever CI runner finishes first — reducing total pipeline time significantly on large test suites.

---

Key APIs

API	Purpose
frappe.set_user("user@example.com")	Switch user context for permission tests
frappe.has_permission("DocType", doc=doc)	Check if current user has access
frappe.get_doc({...}).insert()	Create a test document
frappe.db.get_value("DocType", {"field": "val"})	Fetch a record name
frappe.get_list("DocType", filters=...)	Query list with filters
frappe.in_test	True during test runs — use to skip side effects
frappe.flags.test_xyz_created	Guard flag to avoid duplicate fixture creation

---

Environment Detection

Use frappe.in_test to skip emails, notifications, or external API calls during test runs:

if not frappe.in_test:
    send_notification_to_user()

---

XUnit / CI Output

For Jenkins or GitHub Actions, output JUnit-compatible XML:

bench run-tests --junit-xml-output=/reports/junit_test.xml

The XML format is standard XUnit — works with any CI system that consumes JUnit reports (Jenkins xUnit plugin, GitHub Actions test reporters, GitLab, etc.).

---

📖 Full reference: docs.frappe.io — Unit Testing