| Tags | Tags | No difference in term, but implementation varies. | TestFiesta treats tags as first-class citizens, allowing tagging of nearly everything (cases, runs, plans, etc.). TestRail has limited or no prominent support for tags; custom fields may serve similar but less robust purposes. | |
| Test Plans | Test Plans | No difference; both use "Test Plans". | Significant differences. In TestRail, plans are created by selecting runs and configurations, generating duplicate runs for each configuration (e.g., per browser). In TestFiesta, plans are more complex, allowing selection of multiple runs and configuration groups, manual addition of runs post-creation, individual updates to configurations on a run-by-run basis, and addition of milestones. | |
| Test Executions | Tests | TestFiesta uses "Test Executions" while TestRail refers to them as "Tests" (individual instances within a test run corresponding to a test case). | Generally the same. Both represent instances of executing tests within runs, with statuses like Passed, Failed, etc. | |
| Test Folders | Test Suites and Sections | TestFiesta uses "Test Folders" for organization, while TestRail uses "Test Suites and Sections". | TestFiesta employs simple folders for grouping test cases. TestRail uses a more structured hierarchy with suites (containers for test cases), sections, and sub-sections for specific modules or features. | |
| Configurations | Configurations | Minimal difference; TestFiesta refers to "Configuration Groups" for multiple sets, but the base term is similar. | TestFiesta provides greater flexibility with multiple groups of variables (e.g., browsers, OS) that can be applied and updated per-run within plans. TestRail uses configurations (variables like OS or browsers) primarily to duplicate runs in plans for each combination. | |
| Test Formats | Test Templates | TestFiesta uses "Test Formats" (in some places), while TestRail uses "Test Templates". | TestFiesta allows templates/formats for both test cases and tied results, offering more customization for result capturing. TestRail focuses on predefined structures for test cases (e.g., step-based or exploratory). | |
| Test Milestones | Milestones | TestFiesta uses "Test Milestones", while TestRail uses "Milestones". | Generally the same. Both are key project events or releases (e.g., betas) for tracking progress, with test elements assignable to them. | |
| Test Cases | Test Cases | No difference; both use "Test Cases". | Generally the same. Both consist of prerequisites, steps, and expected results, verifiable by a single tester to confirm specific functionality. | |
| Test Runs | Test Runs | No difference; both use "Test Runs". | Generally the same. Both are executions of test cases or groups, where testers enter results, and can be assigned to milestones or included in plans. | |