5b3a82d621
- The ambiguous character detection is an important security feature to combat against sourcebase attacks (https://trojansource.codes/). - However there are a few problems with the feature as it stands today (i) it's apparantly an big performance hitter, it's twice as slow as syntax highlighting (ii) it contains false positives, because it's reporting valid problems but not valid within the context of a programming language (ambiguous charachters in code comments being a prime example) that can lead to security issues (iii) charachters from certain languages always being marked as ambiguous. It's a lot of effort to fix the aforementioned issues. - Therefore, make it configurable in which context the ambiguous character detection should be run, this avoids running detection in all contexts such as file views, but still enable it in commits and pull requests diffs where it matters the most. Ideally this also becomes an per-repository setting, but the code architecture doesn't allow for a clean implementation of that. - Adds unit test. - Adds integration tests to ensure that the contexts and instance-wide is respected (and that ambigious charachter detection actually work in different places). - Ref: https://codeberg.org/forgejo/forgejo/pulls/2395#issuecomment-1575547 - Ref: https://codeberg.org/forgejo/forgejo/issues/564 |
||
---|---|---|
.. | ||
fixtures | ||
migration-test | ||
schemas | ||
actions_route_test.go | ||
actions_trigger_test.go | ||
admin_config_test.go | ||
admin_user_test.go | ||
api_actions_artifact_test.go | ||
api_activitypub_person_test.go | ||
api_admin_org_test.go | ||
api_admin_test.go | ||
api_block_test.go | ||
api_branch_test.go | ||
api_comment_attachment_test.go | ||
api_comment_test.go | ||
api_feed_plain_text_titles_test.go | ||
api_feed_user_test.go | ||
api_forgejo_root_test.go | ||
api_forgejo_version_test.go | ||
api_fork_test.go | ||
api_gitignore_templates_test.go | ||
api_gpg_keys_test.go | ||
api_helper_for_declarative_test.go | ||
api_httpsig_test.go | ||
api_issue_attachment_test.go | ||
api_issue_config_test.go | ||
api_issue_label_test.go | ||
api_issue_milestone_test.go | ||
api_issue_pin_test.go | ||
api_issue_reaction_test.go | ||
api_issue_stopwatch_test.go | ||
api_issue_subscription_test.go | ||
api_issue_templates_test.go | ||
api_issue_test.go | ||
api_issue_tracked_time_test.go | ||
api_keys_test.go | ||
api_label_templates_test.go | ||
api_license_templates_test.go | ||
api_nodeinfo_test.go | ||
api_notification_test.go | ||
api_oauth2_apps_test.go | ||
api_org_avatar_test.go | ||
api_org_test.go | ||
api_packages_alpine_test.go | ||
api_packages_cargo_test.go | ||
api_packages_chef_test.go | ||
api_packages_composer_test.go | ||
api_packages_conan_test.go | ||
api_packages_conda_test.go | ||
api_packages_container_test.go | ||
api_packages_cran_test.go | ||
api_packages_debian_test.go | ||
api_packages_generic_test.go | ||
api_packages_goproxy_test.go | ||
api_packages_helm_test.go | ||
api_packages_maven_test.go | ||
api_packages_npm_test.go | ||
api_packages_nuget_test.go | ||
api_packages_pub_test.go | ||
api_packages_pypi_test.go | ||
api_packages_rpm_test.go | ||
api_packages_rubygems_test.go | ||
api_packages_swift_test.go | ||
api_packages_test.go | ||
api_packages_vagrant_test.go | ||
api_private_serv_test.go | ||
api_pull_commits_test.go | ||
api_pull_review_test.go | ||
api_pull_test.go | ||
api_releases_test.go | ||
api_repo_archive_test.go | ||
api_repo_avatar_test.go | ||
api_repo_branch_test.go | ||
api_repo_collaborator_test.go | ||
api_repo_edit_test.go | ||
api_repo_file_create_test.go | ||
api_repo_file_delete_test.go | ||
api_repo_file_get_test.go | ||
api_repo_file_helpers.go | ||
api_repo_file_update_test.go | ||
api_repo_files_change_test.go | ||
api_repo_get_contents_list_test.go | ||
api_repo_get_contents_test.go | ||
api_repo_git_blobs_test.go | ||
api_repo_git_commits_test.go | ||
api_repo_git_hook_test.go | ||
api_repo_git_notes_test.go | ||
api_repo_git_ref_test.go | ||
api_repo_git_tags_test.go | ||
api_repo_git_trees_test.go | ||
api_repo_hook_test.go | ||
api_repo_languages_test.go | ||
api_repo_lfs_locks_test.go | ||
api_repo_lfs_migrate_test.go | ||
api_repo_lfs_test.go | ||
api_repo_raw_test.go | ||
api_repo_secrets_test.go | ||
api_repo_tags_test.go | ||
api_repo_teams_test.go | ||
api_repo_test.go | ||
api_repo_topic_test.go | ||
api_settings_test.go | ||
api_team_test.go | ||
api_team_user_test.go | ||
api_token_test.go | ||
api_twofa_test.go | ||
api_user_avatar_test.go | ||
api_user_email_test.go | ||
api_user_follow_test.go | ||
api_user_heatmap_test.go | ||
api_user_info_test.go | ||
api_user_org_perm_test.go | ||
api_user_orgs_test.go | ||
api_user_search_test.go | ||
api_user_secrets_test.go | ||
api_user_star_test.go | ||
api_user_watch_test.go | ||
api_wiki_test.go | ||
attachment_test.go | ||
auth_ldap_test.go | ||
auth_token_test.go | ||
avatar.png | ||
benchmarks_test.go | ||
block_test.go | ||
branches_test.go | ||
change_default_branch_test.go | ||
cmd_forgejo_actions_test.go | ||
cmd_forgejo_test.go | ||
cmd_keys_test.go | ||
compare_test.go | ||
cors_test.go | ||
create_no_session_test.go | ||
csrf_test.go | ||
db_collation_test.go | ||
delete_user_test.go | ||
download_test.go | ||
dump_restore_test.go | ||
editor_test.go | ||
empty_repo_test.go | ||
eventsource_test.go | ||
explore_repos_test.go | ||
forgejo_confirmation_repo_test.go | ||
forgejo_git_test.go | ||
git_clone_wiki_test.go | ||
git_helper_for_declarative_test.go | ||
git_smart_http_test.go | ||
git_test.go | ||
goget_test.go | ||
gpg_git_test.go | ||
html_helper.go | ||
incoming_email_test.go | ||
integration_test.go | ||
issue_test.go | ||
lfs_getobject_test.go | ||
lfs_local_endpoint_test.go | ||
lfs_view_test.go | ||
linguist_test.go | ||
links_test.go | ||
markup_external_test.go | ||
markup_test.go | ||
migrate_test.go | ||
mirror_pull_test.go | ||
mirror_push_test.go | ||
nonascii_branches_test.go | ||
oauth_test.go | ||
org_count_test.go | ||
org_project_test.go | ||
org_team_invite_test.go | ||
org_test.go | ||
private-testing.key | ||
privateactivity_test.go | ||
project_test.go | ||
pull_compare_test.go | ||
pull_create_test.go | ||
pull_diff_test.go | ||
pull_merge_test.go | ||
pull_reopen_test.go | ||
pull_request_task_test.go | ||
pull_review_test.go | ||
pull_status_test.go | ||
pull_update_test.go | ||
README.md | ||
README_ZH.md | ||
release_test.go | ||
rename_branch_test.go | ||
repo_activity_test.go | ||
repo_badges_test.go | ||
repo_branch_test.go | ||
repo_commits_search_test.go | ||
repo_commits_test.go | ||
repo_flags_test.go | ||
repo_fork_test.go | ||
repo_generate_test.go | ||
repo_mergecommit_revert_test.go | ||
repo_migrate_test.go | ||
repo_search_test.go | ||
repo_settings_test.go | ||
repo_tag_test.go | ||
repo_test.go | ||
repo_topic_test.go | ||
repo_watch_test.go | ||
repo_wiki_test.go | ||
repofiles_change_test.go | ||
session_test.go | ||
setting_test.go | ||
signin_test.go | ||
signout_test.go | ||
signup_test.go | ||
ssh_key_test.go | ||
timetracking_test.go | ||
user_avatar_test.go | ||
user_profile_test.go | ||
user_test.go | ||
version_test.go | ||
view_test.go | ||
webfinger_test.go | ||
xss_test.go |
Integration tests
Integration tests can be run with make commands for the appropriate backends, namely:
make test-sqlite
make test-pgsql
make test-mysql
make test-mssql
Make sure to perform a clean build before running tests:
make clean build
Run tests via local act_runner
Run all jobs
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest
Warning: This file defines many jobs, so it will be resource-intensive and therefor not recommended.
Run single job
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -j <job_name>
You can list all job names via:
act_runner exec -W ./.github/workflows/pull-db-tests.yml --event=pull_request --default-actions-url="https://github.com" -i catthehacker/ubuntu:runner-latest -l
Run sqlite integration tests
Start tests
make test-sqlite
Run MySQL integration tests
Setup a MySQL database inside docker
docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql
Run pgsql integration tests
Setup a pgsql database inside docker
docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql
Run mssql integration tests
Setup a mssql database inside docker
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql
Running individual tests
Example command to run GPG test:
For SQLite:
make test-sqlite#GPG
For other databases(replace mssql
to mysql
, or pgsql
):
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG
Setting timeouts for declaring long-tests and long-flushes
We appreciate that some testing machines may not be very powerful and the default timeouts for declaring a slow test or a slow clean-up flush may not be appropriate.
You can either:
- Within the test ini file set the following section:
[integration-tests]
SLOW_TEST = 10s ; 10s is the default value
SLOW_FLUSH = 5S ; 5s is the default value
- Set the following environment variables:
GITEA_SLOW_TEST_TIME="10s" GITEA_SLOW_FLUSH_TIME="5s" make test-sqlite