Constructing Multi Agentic System for Handwritten Reply Analysis

Implementing an automated grading system for handwritten reply sheets utilizing a multi-agent framework streamlines analysis, reduces guide effort, and enhances consistency. A multi-agent system (MAS) consists of autonomous brokers that extract info, grade solutions, and recommend enhancements. By leveraging Handwritten Reply Analysis utilizing Griptape, educators can automate grading, guaranteeing accuracy and effectivity. This method permits lecturers to focus extra on customized suggestions and pupil improvement whereas sustaining equity and reliability in assessments with Handwritten Reply Analysis utilizing Griptape.

Studying Aims

• Perceive the basics, key options, and elements of multi-agent techniques.
• Discover real-world functions the place multi-agent techniques are reworking industries.
• Study Griptape and its function in constructing complicated AI architectures.
• Acquire hands-on expertise in developing a multi-agent system for automated grading with Handwritten Reply Analysis utilizing Griptape.
• Uncover how multi-agent techniques can generate options for enhancing handwritten solutions.

This text was revealed as part of the Knowledge Science Blogathon.

What are Multi Agentic Programs?

Multi-Agent Programs (MAS) are complicated techniques composed of a number of interacting clever brokers, every with its personal specialised capabilities and targets. These brokers will be software program applications, robots, drones, sensors, people, or a mix thereof. MAS leverage collective intelligence, cooperation, and coordination amongst brokers to resolve issues which might be too complicated for a single agent to deal with alone.

Key Traits of Multi-Agent Programs

• Autonomy: Brokers function with a level of independence, making choices based mostly on their native views of the surroundings.
• Decentralization: Management is distributed amongst brokers, permitting the system to operate even when some elements fail.
• Self-Group: Brokers can adapt and arrange themselves based mostly on emergent behaviors, resulting in environment friendly division of labor and battle decision.
• Actual-Time Operation: MAS can reply instantly to altering conditions with out human intervention, making them appropriate for functions like catastrophe response and visitors administration.

MAS can adapt to altering environments by including or eradicating brokers, making them extremely scalable for complicated problem-solving. Decentralized management ensures continued system operation regardless of element failures. MAS can sort out large-scale duties by combining the experience of a number of brokers, outperforming single-agent techniques.

What Constitutes Multi Agentic Programs?

The core elements of multi-agent techniques embody brokers, that are autonomous entities with particular roles and targets, performing because the cognitive core of the system. Duties characterize particular jobs assigned to those brokers, guaranteeing that their efforts are directed in direction of reaching the system’s goals. Instruments lengthen the capabilities of brokers, permitting them to work together with exterior techniques and carry out specialised duties effectively. Moreover, processes define how brokers work together and coordinate actions, guaranteeing that duties are executed in concord. The surroundings supplies the context by which brokers function, influencing their choices and actions.

Lastly, communication protocols allow brokers to share info and negotiate, fostering collaboration or competitors relying on the system’s design. These elements work collectively to allow complicated problem-solving and adaptableness in multi-agent techniques.

Some Key Utility Areas of Multi Agentic System

Multi-agent AI techniques will be extremely helpful in a wide range of functions throughout completely different industries. Listed here are some examples:

• Provide Chain Administration: Multi-agent techniques can optimize logistics by coordinating brokers representing suppliers, producers, distributors, and retailers. These brokers share real-time stock information, predict demand, and handle sources effectively, decreasing stockouts and holding value.
• Healthcare: In healthcare, multi-agent techniques support in illness prediction, affected person allocation, workflow optimization, and customized therapy. Brokers can monitor affected person well being in real-time, notify medical employees about anomalies, and improve communication between sufferers and healthcare suppliers.
• Transportation Programs: Multi-agent techniques enhance visitors stream by coordinating visitors alerts, surveillance cameras, and knowledge techniques. These brokers optimize routes, cut back congestion, and reply to real-time situations like accidents or roadwork.
• Good Grids for Power Administration: Brokers in good grids handle completely different points of electrical energy distribution, from era to good meters. They work collectively to stability vitality provide and demand, combine renewable sources, and keep grid stability

GripTape For Constructing Multi-agent Programs

Griptape is a modular Python framework designed to construct and function multi-agent techniques, that are essential elements of agentic AI techniques. These techniques allow massive language fashions to autonomously deal with complicated duties by integrating a number of AI brokers that work collectively seamlessly. Griptape simplifies the creation of such techniques by offering constructions like brokers, pipelines, and workflows, permitting builders to construct enterprise logic utilizing Python and guaranteeing higher safety, efficiency, and value effectivity.

Core Elements of GripTape

• Agent Construction: Griptape brokers are a part of a modular framework that permits builders to create constructions like brokers, pipelines, and workflows. These constructions are composed of various kinds of duties that allow interplay with massive language fashions (LLMs) and exterior techniques.
• Instruments and Engines: Brokers in Griptape will be geared up with numerous instruments and engines. Specialised elements, known as instruments, carry out particular duties like calculation or summarization, whereas engines, sometimes LLMs, course of prompts and generate outputs.
• Enter and Output Dealing with: Brokers soak up enter straight, which generally is a immediate or a set of arguments. They use their instruments and engines to course of the enter and generate output, which is accessible by the agent’s output attribute.
• Activity Reminiscence and Off-Immediate Knowledge Dealing with: Griptape brokers can deal with off-prompt information, permitting them to course of info past the preliminary immediate, which is helpful for duties that require extra context or information retrieval.
• Drivers for Numerous Use Circumstances: Griptape supplies a spread of drivers that facilitate interactions with completely different LLMs and information processing techniques, making it versatile for numerous functions.

Our Answer: Brokers and Duties

With the growing prevalence of on-line lessons and numerous modes of schooling, there’s a rising scarcity of employees to judge college students’ exams. The sluggish tempo of analysis stays a serious bottleneck in enhancing instructors’ productiveness. Lecturers usually spend a big period of time grading lots of of reply sheets, time that may very well be higher utilized for duties like initiatives, analysis, or straight aiding college students. This situation is especially related as multiple-choice exams should not all the time efficient in assessing a pupil’s understanding of a topic. On this article, we are going to develop a multi-agent system designed to mechanically grade handwritten papers.

Implementing a multi-agent system for automated grading of handwritten reply sheets can considerably streamline the analysis course of for educators. This method makes use of specialised brokers to extract related info from the sheets, assess the solutions based mostly on predefined standards, and even present options for improved responses. By automating these duties, lecturers can deal with extra essential points of schooling, corresponding to customized suggestions and pupil improvement. This know-how can even improve grading consistency and cut back the time spent on guide analysis.

We are going to construct this method utilizing GripTape on Google Colab with T4 GPU (Free Tier).

Palms on Implementation of Computerized Grading of Reply Sheets

Automating handwritten reply analysis with a multi-agent system can enhance accuracy, effectivity, and consistency. By leveraging Griptape, educators can streamline grading, cut back guide effort, and guarantee honest assessments.

Step 1: Putting in and Importing Needed Libraries

The code beneath installs needed dependencies for working with Griptape, Ollama, and Langchain, adopted by importing numerous modules to facilitate creating and managing brokers, duties, and instruments for dealing with completely different information varieties and net searches. It prepares the surroundings to execute a multi-agent system utilizing AI fashions and exterior instruments like file administration and net search.

!pip set up griptape
!sudo apt replace
!sudo apt set up -y pciutils
!pip set up langchain-ollama
!curl -fsSL https://ollama.com/set up.sh | sh
!pip set up ollama==0.4.2
!pip set up "duckduckgo-search>=7.0.1"


import os
from griptape.drivers.immediate.ollama import OllamaPromptDriver
import requests
from griptape.drivers.file_manager.native import LocalFileManagerDriver
from griptape.drivers.immediate.openai import OpenAiChatPromptDriver
from griptape.loaders import ImageLoader
from griptape.constructions import Agent
from griptape.instruments import FileManagerTool, ImageQueryTool
from griptape.duties import PromptTask, StructureRunTask
from griptape.drivers.structure_run.native import LocalStructureRunDriver
from griptape.constructions import Agent, Workflow

from griptape.drivers.web_search.duck_duck_go import DuckDuckGoWebSearchDriver
from griptape.constructions import Agent
from griptape.instruments import PromptSummaryTool, WebSearchTool

Step 2: Beginning Ollama Server & Pulling the Mannequin

The next code begins the ollama server. We additionally pull “minicpm-v” mannequin from ollama in order that this imaginative and prescient mannequin can be utilized to extract textual content from handwritten notes.

import threading
import subprocess
import time

def run_ollama_serve():
  subprocess.Popen(["ollama", "serve"])

thread = threading.Thread(goal=run_ollama_serve)
thread.begin()
time.sleep(5)

!ollama pull minicpm-v

Step 3: Setting OpenAI API Key

import os
os.environ["OPENAI_API_KEY"] = ""

Step 4: Creating an Agent to Learn Handwritten Reply Sheets

The code beneath defines the “image_dir” that shops our photographs or handwritten reply sheets. Additionally a a operate reading_answersheet is outlined that initializes an agent with instruments for managing information and querying photographs utilizing a imaginative and prescient language mannequin (“minicpm-v”). The agent makes use of a file supervisor and a picture question device to course of photographs.

images_dir = os.getcwd()


def reading_answersheet():
      driver = LocalFileManagerDriver(workdir=images_dir)
      return Agent(
          instruments=[
              FileManagerTool(file_manager_driver=driver),
              ImageQueryTool(
                  prompt_driver=OllamaPromptDriver(model="minicpm-v"), image_loader=ImageLoader(file_manager_driver=driver)
              ),
          ]

Step 5: Creating an Agent For Analysis of Solutions utilizing Net 

  This code defines a operate evaluation_answer that creates an agent with instruments for net search utilizing DuckDuckGo.  

def evaluation_answer():
   return Agent(
        instruments=[WebSearchTool(web_search_driver=DuckDuckGoWebSearchDriver()), PromptSummaryTool(off_prompt=False)],
    )

Step 6: Creating Duties For Studying Handwritten Solutions, Scoring them and Suggesting Enhancements

We use this picture for automated analysis. We put it aside in our present working listing as “pattern.jpg”. Its a handwritten reply sheet. This agentic system will first extract the hand written solutions, then consider which solutions are appropriate, rating them and eventually recommend enhancements.

Within the following code blocks, we outline three completely different duties –

• research_task – For extracting texts from handwritten reply sheets. Makes use of “reading_answersheet” agent
• evaluate_task – For scoring the solutions based mostly on the extracted textual content from the handwritten reply sheets. Makes use of “evaluation_answer” agent
• answer_improvement – For suggesting enhancements to the present solutions. Makes use of “evaluation_answer” agent

image_file_name = "pattern.jpg"
group = Workflow()
research_task = StructureRunTask(
        (
            """Extract IN TEXT FORMAT ALL THE LINES GIVEN IMAGE %s"""%(image_file_name),
        ),
        id="analysis",
        structure_run_driver=LocalStructureRunDriver(
            create_structure=reading_answersheet,
        ),
    )

evaluate_task =StructureRunTask(
                (
                  """Confirm whether or not all of the ANSWER containing traces within the TEXT {{ parent_outputs["research"] }} is appropriate and  Rating solely on FACTUAL CORRECTNESS FOR every of those traces on a scale of 1 to 10 based mostly on the correctness of the road. 
             DONT BE too strict in analysis. IGNORE LINES WHICH DO NOT FIT IN THE CONTEXT AND MAY BE JUNK.

                """,
                ),id="consider",
                structure_run_driver=LocalStructureRunDriver(
                    create_structure=evaluation_answer,
                    )
                )

answer_improvement = StructureRunTask(
                (
                  """ADD TO THE PREVIOUS OUTPUT, SUGGESTIONS ON HOW THE ANSWERS IN THE ANSWER containing traces within the TEXT {{ parent_outputs["research"] }} CAN BE IMPROVED BY PROVIDING BETTER OR MORE ACCURATE ANSWERS FOR THOSE ANSWERS THAT DO NOT HAVE 10 SCORE BASED ON THE OUTPUT {{ parent_outputs["evaluate"] }}.
                  DO INCLUDE THE WHOLE OUTPUT FROM THE PREVIOUS AGENT {{ parent_outputs["evaluate"] }}  AS WELL IN THE FINAL OUTPUT.
                
                
                """,
                ),
                structure_run_driver=LocalStructureRunDriver(
                    create_structure=evaluation_answer,
                    )
                )
            

Step 7: Beginning the Workflow and Working it

This code provides duties as youngster duties to a dad or mum workflow (research_task and evaluate_task), then runs the workflow with the duties (research_task, evaluate_task, and answer_improvement) and prints the output of the end result. The workflow orchestrates process execution and captures the ultimate output. Right here, the “answer_improvement” process in added as a baby to each “evaluate_task” and “research_task” in order that it’s run put up the primary two duties.

research_task.add_child(evaluate_task)
evaluate_task.add_child(answer_improvement)
research_task.add_child(answer_improvement)    
group  = Workflow(
    duties=[research_task,evaluate_task,answer_improvement],
)
reply = group.run()
print(reply.output)

Evaluation of Outputs

Enter Picture

Output:

As seen from the output, this agentic system not solely scores every of the solutions but additionally suggests enchancment for every of the solutions which will be very useful to each lecturers and college students, whoever is utilizing this method. 

One other Instance

Hyperlink of Picture

Output

As seen from the output, this agentic system not solely scores every of the solutions but additionally suggests enchancment for every of the solutions. We solely see for the second reply, the system will not be capable of confirm from the net and due to this fact scores it as 5 out of 10. We would want a human intervention for solutions like these on the finish of the loop. Regardless of this, agentic techniques like this may for certain assist lecturers velocity up analysis of lots of of reply sheet.

Conclusion

The implementation of a Handwritten Reply Analysis utilizing Griptape for the automated grading of handwritten reply sheets affords a transformative resolution to the tutorial sector. By automating the grading course of, educators can save precious time, guarantee extra constant evaluations, and deal with offering customized suggestions to college students. Leveraging frameworks like Handwritten Reply Analysis utilizing Griptape additional enhances the flexibleness and scalability of the system, making it a extremely efficient device for modernizing assessments. This method not solely advantages lecturers but additionally improves the general equity and reliability of educational evaluations.

Key Takeaways

• Implementing a multi-agent system (MAS) for automated grading of handwritten reply sheets can drastically cut back guide grading time, permitting educators to deal with extra customized pupil suggestions and improvement.
• Automation helps standardize grading, guaranteeing consistency and equity in assessments, decreasing potential biases inherent in guide evaluations.
• MAS leverage the ability of a number of autonomous brokers working collectively, making it a great resolution for complicated duties like grading, the place brokers deal with duties corresponding to info extraction, evaluation, and suggestions era independently.
• MAS are extremely scalable and adaptable, making them well-suited to deal with massive volumes of information and modify to evolving necessities, corresponding to including or modifying grading standards.
• The Griptape framework simplifies the creation and administration of multi-agent techniques, offering a modular and environment friendly method to develop AI-driven options for real-world challenges like Handwritten Reply Analysis utilizing Griptape.

Ceaselessly Requested Questions

The media proven on this article will not be owned by Analytics Vidhya and is used on the Creator’s discretion.