ChattyG susține un examen de programare uni C/C++ în primul an

ChattyG susține un examen de programare uni C/C++ în primul an

Marca temporală: 3 octombrie 2023 2:58
ChattyG takes a first year uni C/C++ programming exam PlatoBlockchain Data Intelligence. Vertical Search. Ai.

ChatGPT a fost pus la încercare printr-o serie de sarcini de programare C/C++ banale și a trecut – deși nu cu onoruri.

Potrivit unei echipe de cercetare croate, în timp ce studenții din primul an se pot lupta cu unele dintre sarcini, rezultatele [PDF] a arătat că ChatGPT atinge obiective de competență care variau între medie și cea a programatorilor cu experiență. Și, desigur, ca și în cazul tuturor examenelor de facultate, rezultatele pot fi determinate de modul în care sunt formulate întrebările.

Echipa de la Universitatea de Nord a conceput un set de provocări de programare la nivel de boboc, scrise mai întâi în engleză și mai târziu, pentru a vedea dacă nuanțele în mai multe limbi ar afecta rezultatele, croată. Au vrut să vadă nu doar cum codurile ChatGPT, ci și dacă se poate adapta la diferite limbi.

Primul test s-a concentrat pe o sarcină de programare de bază: calcularea celui mai mare divizor comun (GCD) a două numere. La început, bot-ul a arătat câteva limitări în modul în care a decis să abordeze problema, cercetătorii spunând că îi lipsește finețea așteptată de la un programator experimentat. Dar, ca orice student, învață și prin încercări ulterioare, mai ales în versiunea croată, a demonstrat unele îmbunătățiri, dând dovadă de adaptabilitate notabilă.

For example, in one particular task it was challenged to program a basic statistical function in C++. Initially, it made an oversight, using a function that didn’t produce the “corrected” standard deviation as required. But, when the same task was presented in Croatian, the chatbot not only recognized its previous error but worked out a refined solution.

The researchers note that this adaptability mirrors a freshman’s journey: starting with mistakes but showing an ability to learn and enhance their skills with repeated practice and feedback. Awww.

Another task involved a more nuanced problem: identifying numbers within a range based on specific divisibility rules. This was where ChatGPT’s Achilles’ heel became evident. Regardless of language — English or Croatian — ChattyG struggled with negative numbers. Each attempt by ChatGPT led to similar results, pointing towards a consistent issue in its programming logic for this task.

A bonus question demanded precision. ChatGPT was required to craft an input filter, specifically for a defined range of decimal numbers. The AI’s initial solution, when presented in English, was on point, but the next attempts, especially when the task was given in Croatian, revealed some inconsistencies and in some instances, ChatGPT used unnecessary programming constructs. While these didn’t hinder the program’s functionality, it did indicate a lack of optimization. It was as if ChatGPT sometimes took the longer route to a destination, even when a shortcut was available.

Things got more intricate with a task related to arrays. Here, ChatGPT was asked to store numbers and then compute certain statistics like mean value, standard deviation, and identify minimum and maximum values. ChattyG’s performance on this challenge was particularly interesting. Across different tests, it showcased varying strategies. Sometimes, it elegantly solved the problem, offering straightforward solutions. In other attempts, it leaned towards more convoluted methods, even bundling multiple operations into one function.

Toate acestea ridică o întrebare importantă: ChatGPT alege întotdeauna cea mai bună strategie sau, uneori, folosește implicit metodele învățate, dar ineficiente?

The final hurdle for ChatGPT involved basic text processing. It was tasked with removing extra spaces from user input. In its initial English test, ChatGPT’s solution was spot on. However, the Croatian test threw a curveball. Instead of adhering to its effective single-input solution, the AI, for some reason, opted for a more complex approach, demanding multiple inputs. Yet, when researchers revisited this challenge in English, ChatGPT seemed to have learned from its previous misstep, returning to the simpler method.

Overall, the researchers found the responses had a lot in common with those of human freshman programming students. Its solutions often echoed the strategies of experienced programmers but like any student, ChatGPT wasn’t infallible. There were moments of brilliance, but also instances where it seemed to miss the mark entirely .

The real takeaway here is its human freshman-like adaptability: It wasn’t just about getting the right solution; it was about refining, learning, and iterating.

So what’s ChattyG’s final grade?

Din partea cercetătorilor:

“ChatGPT passes the exam with very good grades, outperforming most of our students in the quality of solutions. Furthermore, it solves each task within 20 to 30 seconds and shows the general ability to adapt or change its solutions according to additional demands. However, in some, often simple tasks, it showed the inability to comprehend the logical and mathematical essence of the problem, even after being prompted about its errors several times.” ®

Timestamp-ul:

Mai mult de la Registrul