Are you running security audits on your website? While there are many chat platforms available to choose from, ChatGPT may not meet the accuracy requirements for integration into your security review process. As an AI language model, ChatGPT may pose a risk in terms of data privacy and security. It’s essential to use chat solutions that are safe and secure to protect your business from any potential threats. You can turn to other chat options that meet better accuracy requirements for maintaining the security of your website.
Coinbase Rejects ChatGPT for Security Review Process
Coinbase, a leading cryptocurrency exchange, has announced that it will not be using the popular artificial intelligence tool ChatGPT in its security review process. This decision comes after Coinbase tested the accuracy of ChatGPT by using the tool to assess 20 unnamed ERC-20 tokens. While the results showed promise for quickly assessing smart contract risks, Coinbase found that ChatGPT was not accurate enough to be integrated into its security review processes.
The results of the tests showed that ChatGPT gave eight incorrect answers when compared against the Coinbase security team’s manual review. Five of these incorrect answers were considered the worst-case failure. ChatGPT incorrectly labeled high-risk assets as low-risk, underestimating the risk score. Coinbase noted that underestimating a risk score is far more detrimental than overestimating it.
Furthermore, Coinbase found that ChatGPT was inconsistent when asked the same question repeatedly. The AI tool was also influenced by comments in the code and seemed to default to comments rather than function logic occasionally. Coinbase concluded that ChatGPT does not meet the accuracy requirements to be integrated into its security review processes.
While this experiment represents another example of potential applications of ChatGPT and its latest version, GPT-4, the tool still needs to improve its accuracy before becoming a reliable part of Coinbase’s security review. Crypto enthusiasts have highlighted ChatGPT’s ability to review Ethereum smart contracts and identify vulnerabilities and ways to exploit the code. However, several blockchain developers believe that the tool could assist them in their work but don’t see it replacing humans.
In conclusion, Coinbase’s decision shows the exchange’s commitment to maintaining high standards of security in the ever-evolving crypto landscape. It is crucial for Coinbase to ensure that any tools used in the security review process meet the accuracy requirements to prevent any potential risks.