diff --git a/qa/docs/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md similarity index 97% rename from qa/docs/best_practices.md rename to doc/development/testing_guide/end_to_end/best_practices.md index 62a90563bf0e1ffd4f0a2061bc9816c72bc904aa..89500ef9a909336f2a7160e069c3e5eb19435bb6 100644 --- a/qa/docs/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -35,4 +35,4 @@ Finally, interacting with the application only by its GUI generates a higher rat - Building state through the GUI is time consuming and it's not sustainable as the test suite grows. - When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generate a higher rate of test flakiness. -Now that we are aware of all of it, [let's go create some tests](writing_tests_from_scratch.md#this-document-covers-the-following-items). +Now that we are aware of all of it, [let's go create some tests](quick_start_guide.md). diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end/index.md similarity index 96% rename from doc/development/testing_guide/end_to_end_tests.md rename to doc/development/testing_guide/end_to_end/index.md index fc7aaedca292ac24c49e4fa480049dd46c51ed3b..afd81ff00b2a6789d761b839ba092d212f52c852 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -65,7 +65,7 @@ Below you can read more about how to use it and how does it work. Currently, we are using _multi-project pipeline_-like approach to run QA pipelines. - + <details> <summary>Show mermaid source</summary> @@ -136,6 +136,12 @@ Once you decided where to put [test environment orchestration scenarios] and the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing instance-level scenarios][instance-level scenarios]. +Continued reading: + +- [Quick Start Guide](quick_start_guide.md) +- [Style Guide](style_guide.md) +- [Best Practices](best_practices.md) + ## Where can I ask for help? You can ask question in the `#quality` channel on Slack (GitLab internal) or @@ -149,7 +155,7 @@ you can find an issue you would like to work on in [gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md [quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines [quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines -[review-apps]: ./review_apps.md +[review-apps]: ../review_apps.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario [gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test diff --git a/qa/qa/page/README.md b/doc/development/testing_guide/end_to_end/page_objects.md similarity index 100% rename from qa/qa/page/README.md rename to doc/development/testing_guide/end_to_end/page_objects.md diff --git a/qa/docs/writing_tests_from_scratch.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md similarity index 96% rename from qa/docs/writing_tests_from_scratch.md rename to doc/development/testing_guide/end_to_end/quick_start_guide.md index 06b60a1ef45d102a011aa3cd4d1c3e79b2d35a1d..afe76acf9c97cfb6ee516b101bce71ff7d4be64a 100644 --- a/qa/docs/writing_tests_from_scratch.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -17,10 +17,10 @@ If you don't exactly understand what we mean by **not everything needs to happen - [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`) - [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic - [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods -- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects](./qa/qa/page/README.md) +- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects] - [6.](#6-optimization) Optimizing the test suite - [7.](#7-resources) Using and implementing resources -- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects](./qa/qa/page/README.md) +- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects] ### 0. Are end-to-end tests needed? @@ -111,7 +111,7 @@ end > Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. -> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects](./qa/qa/page/README.md) before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. +> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. Below are the steps that the test covers: @@ -275,7 +275,7 @@ In the `before` block we create all the application state needed for the tests t > A project is created in the background by creating the `issue` resource. -> When [creating the resources](./qa/qa/resource/README.md), notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. +> When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. > What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. @@ -283,7 +283,7 @@ In the `before` block we create all the application state needed for the tests t ### 6. Optimization -As already mentioned in the [best practices](./BEST_PRACTICES.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible. +As already mentioned in the [best practices](best_practices.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible. > Note that end-to-end tests are slow to run and so they can have several actions and assertions in a single test, which helps us get feedback from the tests sooner. In comparison, unit tests are much faster to run and can exercise every little piece of the application in isolation, and so they usually have only one assertion per test. @@ -339,7 +339,7 @@ To address point 1, we changed the test implementation from two `it` blocks into **Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation. -You can think of [resources](qa/qa/resource/README.md) as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. +You can think of [Resources] as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc. @@ -439,7 +439,7 @@ Page Objects are used in end-to-end tests for maintenance reasons, where a page' > Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`). -Take a look at [this document for a detailed explanation about Page Objects](./qa/page/README.md). +Take a look at the [Page Objects] documentation. Now, let's go back to our example. @@ -580,3 +580,6 @@ As you might remember, in the Issue Page Object we call this method like this: ` ___ With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* + +[Page Objects]: page_objects.md +[Resources]: resources.md diff --git a/qa/qa/resource/README.md b/doc/development/testing_guide/end_to_end/resources.md similarity index 97% rename from qa/qa/resource/README.md rename to doc/development/testing_guide/end_to_end/resources.md index 2c8859b6599931877c349c50e8763f6c941001bb..1e32db4f63355fb2c6d4cf4a8519a119a0c9e246 100644 --- a/qa/qa/resource/README.md +++ b/doc/development/testing_guide/end_to_end/resources.md @@ -5,11 +5,11 @@ be created via the API or the CLI. ## How to properly implement a resource class? -All resource classes should inherit from [`Resource::Base`](./base.rb). +All resource classes should inherit from `Resource::Base`. There is only one mandatory method to implement to define a resource class. This is the `#fabricate!` method, which is used to build the resource via the -browser UI. Note that you should only use [Page objects](../page/README.md) to +browser UI. Note that you should only use [Page objects](page_objects.md) to interact with a Web page in this method. Here is an imaginary example: @@ -74,7 +74,7 @@ module QA end ``` -The [`Project` resource](./project.rb) is a good real example of Browser +The `Project` resource is a good real example of Browser UI and API implementations. #### Resource attributes diff --git a/qa/docs/guidelines.md b/doc/development/testing_guide/end_to_end/style_guide.md similarity index 94% rename from qa/docs/guidelines.md rename to doc/development/testing_guide/end_to_end/style_guide.md index cd4b939fd719ba61538c7d1ab00bd50d2523b7f0..0272e1810f2cc934790f8cc17264235f5c8a5b19 100644 --- a/qa/docs/guidelines.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -1,6 +1,6 @@ -# Style guide for writing E2E tests +# Style guide for writing end-to-end tests -This document describes the conventions used at GitLab for writing E2E tests using the GitLab QA project. +This document describes the conventions used at GitLab for writing End-to-end (E2E) tests using the GitLab QA project. ## `click_` versus `go_to_` @@ -45,7 +45,7 @@ Notice that in the above example, before clicking the `:operations_environments_ > We can create these methods as helpers to abstract multi-step navigation. -### Element Naming Convention +### Element naming convention When adding new elements to a page, it's important that we have a uniform element naming convention. @@ -69,7 +69,8 @@ We follow a simple formula roughly based on hungarian notation. #### Examples -**Good** +**Good** + ```ruby view '...' do element :edit_button @@ -80,7 +81,8 @@ view '...' do end ``` -**Bad** +**Bad** + ```ruby view '...' do # `_confirmation` should be `_field`. what sort of confirmation? a checkbox confirmation? no real way to disambiguate. diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index ecad9ba48a3e4114b15d1861e250d073c8fd9e03..93ee2a6371ab1870f140b693582932ef01a6fc02 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -71,7 +71,7 @@ Everything you should know about how to test Rake tasks. --- -## [End-to-end tests](end_to_end_tests.md) +## [End-to-end tests](end_to_end/index.md) Everything you should know about how to run end-to-end tests using [GitLab QA][gitlab-qa] testing framework. diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md index 30d861d7d68bdb2a9fdd6c58fbcc98119b7370a9..c9d3238fbe90c6414d70f4abf358addc81dd94dd 100644 --- a/doc/development/testing_guide/smoke.md +++ b/doc/development/testing_guide/smoke.md @@ -17,7 +17,7 @@ Currently, our suite consists of this basic functionality coverage: Smoke tests have the `:smoke` RSpec metadata. -See [End-to-end Testing](./end_to_end_tests.md) for more details about +See [End-to-end Testing](end_to_end/index.md) for more details about end-to-end tests. --- diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index b5155b6b7fa3ec7c614d735404f59dfcc1f43012..e1ce4d3b7d1f10585f73073a04203e388793160f 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -178,7 +178,7 @@ Every new feature should come with a [test plan]. | ---------- | -------------- | ----- | | `qa/qa/specs/features/` | [Capybara] + [RSpec] + Custom QA framework | Tests should be placed under their corresponding [Product category] | -> See [end-to-end tests](end_to_end_tests.md) for more information. +> See [end-to-end tests](end_to_end/index.md) for more information. Note that `qa/spec` contains unit tests of the QA framework itself, not to be confused with the application's [unit tests](#unit-tests) or diff --git a/qa/README.md b/qa/README.md index f75205133e65aea6365131db96d3e89a19c1dffa..ef6f202464d735545724bda51e2c1178268ab786 100644 --- a/qa/README.md +++ b/qa/README.md @@ -1,6 +1,6 @@ # GitLab QA - End-to-end tests for GitLab -This directory contains [end-to-end tests](doc/development/testing_guide/end_to_end_tests.md) +This directory contains [end-to-end tests](../../../doc/development/testing_guide/end_to_end/index.md) for GitLab. It includes the test framework and the tests themselves. The tests can be found in `qa/specs/features` (not to be confused with the unit @@ -29,7 +29,7 @@ verify coupling between page objects implemented as a part of GitLab QA and corresponding views / partials / selectors in CE / EE. Whenever `qa:selectors` job fails in your merge request, you are supposed to -fix [page objects](qa/page/README.md). You should also trigger end-to-end tests +fix [page objects](../doc/development/testing_guide/end_to_end/page_objects.md). You should also trigger end-to-end tests using `package-and-qa` manual action, to test if everything works fine. ## How can I use it? @@ -49,10 +49,10 @@ will need to [modify your GDK setup](https://gitlab.com/gitlab-org/gitlab-qa/blo ### Writing tests -- [Writing tests from scratch tutorial](docs/writing_tests_from_scratch.md) - - [Best practices](docs/best_practices.md) - - [Using page objects](qa/page/README.md) - - [Guidelines](docs/guidelines.md) +- [Writing tests from scratch tutorial](../doc/development/testing_guide/end_to_end/quick_start_guide.md) + - [Best practices](../doc/development/testing_guide/best_practices.md) + - [Using page objects](../doc/development/testing_guide/end_to_end/page_objects.md) + - [Guidelines](../doc/development/testing_guide/index.md) ### Running specific tests