Security-Focused Review Guide
Background
HackerOne offers a limited scope version of PullRequest dedicated to code security scrutiny. Certain customers require that feedback escalated to their development team is security or security-adjacent in nature. This means that any feedback falling outside of those categories is out of scope and should therefore not be raised.
This solution aims to address the growing need for developer-friendly security tools, providing on-demand security expertise to accelerate development cycles and reduce vulnerabilities before production. This enhances HackerOne's continuous vulnerability testing framework by adding a robust code review component, ensuring a more comprehensive security posture from development through production.
Customer Goals
Avoid alert fatigue from noisy code scanners and AI. Human experts will raise only the concerns that matter, reducing the chance for false positives while also adding a layer of scrutiny before Merge.
Give developers access to the security context they need to ship secure code, faster—without the false positives and bottlenecks.
Reviewer Expectations
Keep feedback related to security concerns
Raise feedback only on valid issues or bring attention to issues that could be valid, but require some clarification from the development team to determine validity
Focus on areas highlighted by our security automation and AI hot spots (See next section for more information on these), only raising issues that are valid
Focus on other areas not highlighted by our automation that deserve additional scrutiny per your expert judgement
How to Complete a Security-Focused Review
The reviewer interface for this type of review has been designed to ensure that the reviewers engaged have the tools they need to do the job more easily and with the right expectations. The reviewer experience for security-specific reviews is different from our more generalized code review offering that reviewers may be accustomed to. Completing a review can be done with the following three steps in mind.
Step 1: Verify Automated Findings
Review automated results raised by various automated SAST tools provided. The Automation Results Checklist allows reviewers to skip directly to each automated security rule triggered.
For any automated results that are found to be invalid or false positives, ignore them. Those that are found to be valid should be escalated using the Convert to a comment button. If you are unsure of the validity of an automated finding or you are missing some relevant context, raise the finding with a clarifying question and provide as much information as to why you think it could be a valid risk for the code author to consider.
Review any code specifically outlined in pink. Our AI engine attempts to find risky areas of code and highlights sections to draw attention to them. As with other automated findings, the expectation should be for reviewers to only raise issues related to verified risk using the power of human experts! To report any valid findings here, simply comment inline and provide thoughtful details for the code author to engage with.
In the end, you bring subject matter expertise to review our customer's code. If you feel there are other areas that deserve additional scrutiny in a security context, we encourage going outside of the automation. In the end, this should be a faster and more focused review.
Do you find yourself needing more context on the code? Try out our context engine HAI. Simply ask HAI a question about the codebase and we'll generate some context to help with the review. Context generated is based on only the repository in focus to help reviewers learn about the codebase more quickly and dynamically than just with our search feature.
Step 2: Submit the Review
Once the review is complete, make sure to submit it using the Review button in the top right of the review window. Submitting a summary comment is optional and should not be used if nothing of note was found. If something was discovered during the review, feel free to expand on it and leave a summary of one or multiple pieces of feedback found.
Step 3: Follow Up Reviews
Any new commits related to security feedback raised will receive another review. If it makes sense, use the relevant comment thread to engage further to suggest additional changes or indicate that changes made will resolve the issue. This process will continue until the change is ultimately merged or closed by the customer.
Last updated
