اپنی مرضی کے عینک کے ساتھ اچھی طرح سے تعمیر شدہ IDP حل بنائیں – حصہ 4: کارکردگی کی کارکردگی | ایمیزون ویب سروسز

When a customer has a production-ready ذہین دستاویز پروسیسنگ (IDP) workload, we often receive requests for a Well-Architected review. To build an enterprise solution, developer resources, cost, time and user-experience have to be balanced to achieve the desired business outcome. The AWS اچھی طرح سے آرکیٹیکٹڈ فریم ورک۔ provides a systematic way for organizations to learn operational and architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable workloads in the cloud.

The IDP Well-Architected Custom Lens follows the AWS Well-Architected Framework, reviewing the solution with six pillars with the granularity of a specific AI or machine learning (ML) use case, and providing the guidance to tackle common challenges. The IDP Well-Architected Custom Lens in the اچھی طرح سے تعمیر شدہ ٹول contains questions regarding each of the pillars. By answering these questions, you can identify potential risks and resolve them by following your improvement plan.

This post focuses on the Performance Efficiency pillar of the IDP workload. We dive deep into designing and implementing the solution to optimize for throughput, latency, and overall performance. We start with discussing some common indicators that you should conduct a Well-Architected review, and introduce the fundamental approaches with design principles. Then we go through each focus area from a technical perspective.

To follow along with this post, you should be familiar with the previous posts in this series (حصہ 1 اور حصہ 2) and the guidelines in AWS پر ذہین دستاویز پروسیسنگ کے لیے رہنمائی. These resources introduce common AWS services for IDP workloads and suggested workflows. With this knowledge, you’re now ready to learn more about productionizing your workload.

Common indicators

The following are common indicators that you should conduct a Well-Architected Framework review for the Performance Efficiency pillar:

• زیادہ تاخیر – When the latency of optical character recognition (OCR), entity recognition, or the end-to-end workflow takes longer than your previous benchmark, this may be an indicator that the architecture design doesn’t cover load testing or error handling.
• Frequent throttling – You may experience throttling by AWS services like ایمیزون ٹیکسٹ due to request limits. This means that the architecture needs to be adjusted by reviewing the architecture workflow, synchronous and asynchronous implementation, transactions per second (TPS) calculation, and more.
• Debugging difficulties – When there’s a document process failure, you may not have an effective way to identify where the error is located in the workflow, which service it’s related to, and why the failure occurred. This means the system lacks visibility into logs and failures. Consider revisiting the logging design of the telemetry data and adding infrastructure as code (IaC), such as document processing pipelines, to the solution.

انڈیکیٹر	Description	Architectural Gap
اعلی دیر	OCR, entity recognition, or end-to-end workflow latency exceeds previous benchmark	لوڈ ٹیسٹنگ اغلاط کی درستگی
Frequent Throttling	Throttling by AWS services like Amazon Textract due to request limits	Sync vs Async TPS calculation
Hard to Debug	No visibility into location, cause, and reason for document processing failures	Logging Design Document Processing Pipelines

اصول ڈیزائن کریں

In this post, we discuss three design principles: delegating complex AI tasks, IaC architectures, and serverless architectures. When you encounter a trade-off between two implementations, you can revisit the design principles with the business priorities of your organization so that you can make decisions effectively.

• Delegating complex AI tasks – You can enable faster AI adoption in your organization by offloading the ML model development lifecycle to managed services and taking advantage of the model development and infrastructure provided by AWS. Rather than requiring your data science and IT teams to build and maintain AI models, you can use pre-trained AI services that can automate tasks for you. This allows your teams to focus on higher-value work that differentiates your business, while the cloud provider handles the complexity of training, deploying, and scaling the AI models.
• IaC architectures – When running an IDP solution, the solution includes multiple AI services to perform the end-to-end workflow chronologically. You can architect the solution with workflow pipelines using AWS اسٹیپ فنکشنز to enhance fault tolerance, parallel processing, visibility, and scalability. These advantages can enable you to optimize the usage and cost of underlying AI services.
• بے سرور آرکیٹیکچرز – IDP is often an event-driven solution, initiated by user uploads or scheduled jobs. The solution can be horizontally scaled out by increasing the call rates for the AI services, او ڈبلیو ایس لامبڈا۔, and other services involved. A serverless approach provides scalability without over-provisioning resources, preventing unnecessary expenses. The monitoring behind the serverless design assists in detecting performance issues.

With these three design principles in mind, organizations can establish an effective foundation for AI/ML adoption on cloud platforms. By delegating complexity, implementing resilient infrastructure, and designing for scale, organizations can optimize their AI/ML solutions.

In the following sections, we discuss how to address common challenges in regards to technical focus areas.

فوکس علاقوں

When reviewing performance efficiency, we review the solution from five focus areas: architecture design, data management, error handling, system monitoring, and model monitoring. With these focus areas, you can conduct an architecture review from different aspects to enhance the effectivity, observability, and scalability of the three components of an AI/ML project, data, model, or business goal.

آرکیٹیکچر ڈیزائن

By going through the questions in this focus area, you will review the existing workflow to see if it follows best practices. The suggested workflow provides a common pattern that organizations can follow and prevents trial-and-error costs.

کی بنیاد پر مجوزہ فن تعمیر, the workflow follows the six stages of data capture, classification, extraction, enrichment, review and validation, and consumption. In the common indicators we discussed earlier, two out of three come from architecture design problems. This is because when you start a project with an improvised approach, you may meet project restraints when trying to align your infrastructure to your solution. With the architecture design review, the improvised design can be decoupled as stages, and each of them can be reevaluated and reordered.

You can save time, money, and labor by implementing درجہ بندی in your workflow, and documents go to downstream applications and APIs based on document type. This enhances the observability of the document process and makes the solution straightforward to maintain when adding new document types.

ڈیٹا مینجمنٹ

Performance of an IDP solution includes latency, throughput, and the end-to-end user experience. How to manage the document and its extracted information in the solution is the key to data consistency, security, and privacy. Additionally, the solution must handle high data volumes with low latency and high throughput.

When going through the questions of this focus area, you will review the document workflow. This includes data ingestion, data preprocessing, converting documents to document types accepted by Amazon Textract, handling incoming document streams, routing documents by type, and implementing access control and retention policies.

For example, by storing a document in the different processed phases, you can reverse processing to the previous step if needed. The data lifecycle ensures the reliability and compliance for the workload. By using the Amazon Textract Service Quotas Calculator (see the following screenshot), asynchronous features on Amazon Textract, Lambda, Step Functions, ایمیزون سادہ قطار سروس (ایمیزون ایس کیو ایس)، اور ایمیزون سادہ نوٹیفکیشن سروس (Amazon SNS), organizations can automate and scale document processing tasks to meet specific workload needs.

اغلاط کی درستگی

Robust error handling is critical for tracking the document process status, and it provides the operation team time to react to any abnormal behaviors, such as unexpected document volumes, new document types, or other unplanned issues from third-party services. From the organization’s perspective, proper error handling can enhance system uptime and performance.

You can break down error handling into two key aspects:

• AWS service configuration – You can implement retry logic with exponential backoff to handle transient errors like throttling. When you start processing by calling an asynchronous Start* operation, such as StartDocumentTextDetection, you can specify that the completion status of the request is published to an SNS topic in the NotificationChannel configuration. This helps you avoid throttling limits on API calls due to polling the Get* APIs. You can also implement alarms in ایمیزون کلاؤڈ واچ and triggers to alert when unusual error spikes occur.
• Error report enhancement – This includes detailed messages with an appropriate level of detail by error type and descriptions of error handling responses. With the proper error handling setup, systems can be more resilient by implementing common patterns like automatically retrying intermittent errors, using circuit breakers to handle cascading failures, and monitoring services to gain insight into errors. This allows the solution to balance between retry limits and prevents never-ending circuit loops.

ماڈل کی نگرانی

The performance of ML models is monitored for degradation over time. As data and system conditions change, the model performance and efficiency metrics are tracked to ensure retraining is performed when needed.

The ML model in an IDP workflow can be an OCR model, entity recognition model, or classification model. The model can come from an AWS AI service, an open source model on ایمیزون سیج میکر, ایمیزون بیڈرک, or other third-party services. You must understand the limitations and use cases of each service in order to identify ways to improve the model with human feedback and enhance service performance over time.

A common approach is using service logs to understand different levels of accuracy. These logs can help the data science team identify and understand any need for model retraining. Your organization can choose the retraining mechanism—it can be quarterly, monthly, or based on science metrics, such as when accuracy drops below a given threshold.

The goal of monitoring is not just detecting issues, but closing the loop to continuously refine models and keep the IDP solution performing as the external environment evolves.

سسٹم کی نگرانی

After you deploy the IDP solution in production, it’s important to monitor key metrics and automation performance to identify areas for improvement. The metrics should include business metrics and technical metrics. This allows the company to evaluate the system’s performance, identify issues, and make improvements to models, rules, and workflows over time to increase the automation rate to understand the operational impact.

On the business side, metrics like extraction accuracy for important fields, overall automation rate indicating the percentage of documents processed without human intervention, and average processing time per document are paramount. These business metrics help quantify the end-user experience and operational efficiency gains.

Technical metrics including error and exception rates occurring throughout the workflow are essential to track from an engineering perspective. The technical metrics can also monitor at each level from end to end and provide a comprehensive view of a complex workload. You can break the metrics down into different levels, such as solution level, end-to-end workflow level, document type level, document level, entity recognition level, and OCR level.

Now that you have reviewed all the questions in this pillar, you can assess the other pillars and develop an improvement plan for your IDP workload.

نتیجہ

In this post, we discussed common indicators that you may need to perform a Well-Architected Framework review for the Performance Efficiency pillar for your IDP workload. We then walked through design principles to provide a high-level overview and discuss the solution goal. By following these suggestions in reference to the IDP Well-Architected Custom Lens and by reviewing the questions by focus area, you should now have a project improvement plan.

---

مصنفین کے بارے میں

میا چانگ ایمیزون ویب سروسز کے لیے ایم ایل اسپیشلسٹ سلوشنز آرکیٹیکٹ ہے۔ وہ EMEA میں صارفین کے ساتھ کام کرتی ہے اور اطلاقی ریاضی، کمپیوٹر سائنس، اور AI/ML میں اپنے پس منظر کے ساتھ کلاؤڈ پر AI/ML ورک بوجھ چلانے کے بہترین طریقوں کا اشتراک کرتی ہے۔ وہ NLP سے متعلق مخصوص کام کے بوجھ پر توجہ مرکوز کرتی ہے، اور ایک کانفرنس اسپیکر اور کتاب کے مصنف کے طور پر اپنا تجربہ شیئر کرتی ہے۔ اپنے فارغ وقت میں، وہ پیدل سفر، بورڈ گیمز اور کافی پینے سے لطف اندوز ہوتی ہے۔

برجیش پتی۔ AWS میں ایک انٹرپرائز سلوشنز آرکیٹیکٹ ہے۔ اس کی بنیادی توجہ انٹرپرائز صارفین کو ان کے کام کے بوجھ کے لیے کلاؤڈ ٹیکنالوجیز کو اپنانے میں مدد کرنا ہے۔ اس کا ایپلی کیشن ڈویلپمنٹ اور انٹرپرائز آرکیٹیکچر میں پس منظر ہے اور اس نے مختلف صنعتوں جیسے کھیل، مالیات، توانائی اور پیشہ ورانہ خدمات کے صارفین کے ساتھ کام کیا ہے۔ اس کی دلچسپیوں میں سرور لیس فن تعمیر اور AI/ML شامل ہیں۔

روئی کارڈوسو Amazon Web Services (AWS) میں ایک پارٹنر سلوشن آرکیٹیکٹ ہے۔ وہ AI/ML اور IoT پر توجہ مرکوز کر رہا ہے۔ وہ AWS شراکت داروں کے ساتھ کام کرتا ہے اور AWS میں حل تیار کرنے میں ان کی مدد کرتا ہے۔ کام نہ کرنے پر اسے سائیکل چلانے، پیدل سفر کرنے اور نئی چیزیں سیکھنے میں مزہ آتا ہے۔

ٹم کونڈیلو ایمیزون ویب سروسز (AWS) میں ایک سینئر مصنوعی ذہانت (AI) اور مشین لرننگ (ML) ماہر حل آرکیٹیکٹ ہے۔ اس کا فوکس قدرتی زبان کی پروسیسنگ اور کمپیوٹر ویژن ہے۔ ٹم کو گاہک کے خیالات لینے اور انہیں قابل توسیع حل میں تبدیل کرنے میں مزہ آتا ہے۔

شیری ڈنگ ایمیزون ویب سروسز (AWS) میں ایک سینئر مصنوعی ذہانت (AI) اور مشین لرننگ (ML) ماہر حل آرکیٹیکٹ ہے۔ وہ کمپیوٹر سائنس میں پی ایچ ڈی کی ڈگری کے ساتھ مشین لرننگ کا وسیع تجربہ رکھتی ہیں۔ وہ بنیادی طور پر پبلک سیکٹر کے صارفین کے ساتھ مختلف AI/ML متعلقہ کاروباری چیلنجز پر کام کرتی ہے، جس سے انہیں AWS کلاؤڈ پر مشین لرننگ کے سفر کو تیز کرنے میں مدد ملتی ہے۔ گاہکوں کی مدد نہ کرنے پر، وہ بیرونی سرگرمیوں سے لطف اندوز ہوتی ہے۔

سوین وانگ AWS میں ایک AI/ML ماہر حل آرکیٹیکٹ ہے۔ اس کے پاس مشین لرننگ، فنانشل انفارمیشن سروس اور اکنامکس میں بین الضابطہ تعلیم کا پس منظر ہے، اس کے ساتھ ساتھ ڈیٹا سائنس اور مشین لرننگ ایپلی کیشنز بنانے کے تجربے کے ساتھ جو حقیقی دنیا کے کاروباری مسائل کو حل کرتی ہیں۔ وہ گاہکوں کو صحیح کاروباری سوالات کی نشاندہی کرنے اور صحیح AI/ML حل تیار کرنے میں مدد کرتی ہے۔ اپنے فارغ وقت میں، وہ گانا اور کھانا پکانا پسند کرتی ہے۔

• SEO سے چلنے والا مواد اور PR کی تقسیم۔ آج ہی بڑھا دیں۔
• پلیٹو ڈیٹا ڈاٹ نیٹ ورک ورٹیکل جنریٹو اے آئی۔ اپنے آپ کو بااختیار بنائیں۔ یہاں تک رسائی حاصل کریں۔
• پلیٹوآئ اسٹریم۔ ویب 3 انٹیلی جنس۔ علم میں اضافہ۔ یہاں تک رسائی حاصل کریں۔
• پلیٹو ای ایس جی۔ کاربن، کلین ٹیک، توانائی ، ماحولیات، شمسی، ویسٹ مینجمنٹ یہاں تک رسائی حاصل کریں۔
• پلیٹو ہیلتھ۔ بائیوٹیک اینڈ کلینیکل ٹرائلز انٹیلی جنس۔ یہاں تک رسائی حاصل کریں۔
• ماخذ: https://aws.amazon.com/blogs/machine-learning/build-well-architected-idp-solutions-with-a-custom-lens-part-4-performance-efficiency/