Gru.ai Ranks First in OpenAI's Latest SWE-Bench Verified Evaluation

September 10, 2024 12:00 AM AEST | By Cision

Gru.ai Ranks First in OpenAI's Latest SWE-Bench Verified Evaluation — Image source: Kalkine Media

Recent News

Meihua International Medical Technologies Co., Limited Reports Unaudit...

Bloomberg Expands Climate Solutions To Enhance Corporate Transition Ri...

Fed's Bowman voted against jumbo cut to avoid signaling victory on inf...

Schneider Electric and the Schneider Electric Foundation Partner With...

SAN FRANCISCO, Sept. 9, 2024 /PRNewswire/ -- On 3rd September, Gru.ai ranked first with a high score of 45.2% in the latest data released by SWE-Bench Verified Evaluation, the authoritative standard for AI model evaluation. The SWE-Bench Verified, a reliable evaluation of AI models' ability to solve real-world software issues, was a Benchmark of collaboration between OpenAI and SWE.

Bug Fix Gru, one of four agents provided by Gru, participated in the SWE-Bench Verified evaluation. According to Gru's team blog, providing Bug Fix Gru with a comprehensive operating environment and a wealth of development tools laid the foundation for their high score. Enhancements in workflow, multimodal support, and the addition of Rag capabilities effectively boosted the score. Notably, the Gru team emphasized that they have an evaluation process in place to assess the impact of any changes.

Gru.ai, a company that builds AI developers, provides four types of software engineering agents:

Assistant Gru: Helps users solve standalone technical issues, which is now in public use.
Test Gru: Generates unit test code automatically
Bug Fix Gru: Fixes bugs based on user issues automatically
Babel Gru: Assists in building end-to-end projects

Gru.ai previously secured a $5.5 million angel investment. Alongside Gru, several other firms in the sector, including Devin, Factory, Cosine.sh and Codium.ai, have also announced their funding details. As large-scale model capabilities mature, the coding agent field is experiencing a surge of investment and innovation, indicating a bright future for this evolving industry.

Disclaimer

The content, including but not limited to any articles, news, quotes, information, data, text, reports, ratings, opinions, images, photos, graphics, graphs, charts, animations and video (Content) is a service of Kalkine Media Pty Ltd (“Kalkine Media, we or us”), ACN 629 651 672 and is available for personal and non-commercial use only. The principal purpose of the Content is to educate and inform. The Content does not contain or imply any recommendation or opinion intended to influence your financial decisions and must not be relied upon by you as such. Some of the Content on this website may be sponsored/non-sponsored, as applicable, but is NOT a solicitation or recommendation to buy, sell or hold the stocks of the company(s) or engage in any investment activity under discussion. Kalkine Media is neither licensed nor qualified to provide investment advice through this platform. Users should make their own enquiries about any investments and Kalkine Media strongly suggests the users to seek advice from a financial adviser, stockbroker or other professional (including taxation and legal advice), as necessary.
The content published on Kalkine Media also includes feeds sourced from third-party providers. Kalkine does not assert any ownership rights over the content provided by these third-party sources. The inclusion of such feeds on the Website is for informational purposes only. Kalkine does not guarantee the accuracy, completeness, or reliability of the content obtained from third-party feeds. Furthermore, Kalkine Media shall not be held liable for any errors, omissions, or inaccuracies in the content obtained from third-party feeds, nor for any damages or losses arising from the use of such content.
Kalkine Media hereby disclaims any and all the liabilities to any user for any direct, indirect, implied, punitive, special, incidental or other consequential damages arising from any use of the Content on this website, which is provided without warranties. The views expressed in the Content by the guests, if any, are their own and do not necessarily represent the views or opinions of Kalkine Media. Some of the images/music that may be used on this website are copyrighted to their respective owner(s). Kalkine Media does not claim ownership of any of the pictures displayed/music used on this website unless stated otherwise. The images/music that may be used on this website are taken from various sources on the internet, including paid subscriptions or are believed to be in public domain. We have made reasonable efforts to accredit the source wherever it was indicated as or found to be necessary.

This disclaimer is subject to change without notice. Users are advised to review this disclaimer periodically for any updates or modifications.