CAIC4 Flashcards

(51 cards)

1
Q

What is the general rule regarding data for model training?

A

The more data you have, the stronger performance you should have across the board.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What teams might be involved in the deployment process of AI/ML models?

A

Data scientists, validation teams, engineering teams, ML engineers.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What factors should be analyzed in the deployment environment?

A
  • Best way to access the model (API/UI)
  • Frequency of calls
  • Required hardware (GPUs/CPUs/memory)
  • Data feeding method.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the importance of training end users on AI/ML models?

A

Interpretability and effective communication of results.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are the components of continuous maintenance in AI/ML?

A
  • Continuous integration
  • Continuous delivery
  • Continuous training
  • Continuous monitoring.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Why is managing model performance post-deployment crucial?

A

It is a highly iterative, never-ending process of model maintenance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What metrics should be monitored to ensure model performance?

A
  • Accuracy
  • Recall
  • Precision
  • F-score
  • R-squared.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is concept drift in the context of AI/ML?

A

Changes in customer expectations or data reporting that affect model predictions.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is the significance of data hygiene in model performance?

A

Poor data hygiene can lead to degraded model performance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What should companies do to ensure ethical AI practices?

A

Monitor models for accuracy and drift; train all resources on ethics.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What legislation was signed by President Biden to enhance AI accountability?

A
  • Artificial Intelligence for the Military Act of 2021
  • AICT Act of 2021.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the four key areas outlined by the European Commission for ethical AI?

A
  • Human-centric allocation of functions
  • Prevention of harm
  • Equal and just distribution
  • Transparency in output and decision-making.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

True or False: Continuous monitoring of AI/ML models is optional.

A

False.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Fill in the blank: Continuous monitoring ensures that the outputs generated by the model are _______.

A

[effective].

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the risk associated with poorly monitored AI/ML systems?

A

They can cause real-world harm and damage company integrity.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How can ethical AI practices benefit product marketing?

A

They allow for confident marketing without fear of retribution.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What is a common misconception about AI/ML model maintenance?

A

67% of AI users do not monitor their models for accuracy or drift.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is the potential consequence of not maintaining AI/ML documentation?

A

Loss of historical knowledge and inadequate resources.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is Ethical AI?

A

Ethical AI refers to the application of ethical practices throughout every step of AI/ML product development.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is Generative AI?

A

Generative AI is a type of AI that can generate new content based on the data it has been trained on.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What does AI stand for?

A

Artificial Intelligence

22
Q

What is Machine Learning (ML)?

A

A subset of AI that autonomously learns from historical data and makes predictions based on acquired patterns.

23
Q

How does Deep Learning differ from Machine Learning?

A

Deep Learning uses artificial deep neural networks to emulate cognitive abilities, refining outputs through multiple layers.

24
Q

What is the significance of a Well Architected Enterprise Generative AI Framework?

A

It offers a structured methodology to mitigate business and technical challenges while adopting Generative AI capabilities.

25
List the six pillars of the proposed Generative AI framework.
* Operational Excellence * Security * Privacy * Compliance * Cost Optimization * Model Management
26
What is the focus of the Operational Excellence pillar?
Identifying the best model or mix of models for a use case.
27
What role does the Compliance pillar play in the Generative AI framework?
It enforces secure archival processes for generated content for auditing and regulatory requirements.
28
True or False: ChatGPT is a Large Language Model (LLM).
False
29
What are some examples of tasks a foundation model can perform?
* Summarizing documents * Generating content * Paraphrasing * Classifying content from images * Producing code * Solving complex math problems * Translating languages
30
What is the difference between proprietary LLMs and open-source LLMs?
* Proprietary LLMs are developed by specialized AI providers and are tightly governed * Open-source LLMs are maintained by the community and are publicly accessible
31
Fill in the blank: A ________ model is at the heart of any Enterprise Generative AI architecture.
Large Language Model (LLM)
32
What is the importance of identifying the correct generative model?
It lays the foundation for a scalable product and assists in focusing on operational aspects.
33
What are the two building blocks that empower the security pillar?
* Content Moderation * AI Guardrails
34
What does the content moderation component help with?
* Checking unwanted user input * Filtering potentially unsafe dialogue
35
What challenge does the Compliance pillar address?
It ensures that the generated content meets regulatory requirements for auditing.
36
What is the main purpose of content moderation in AI products?
To ensure both the content generated by the model and the input content from users maintain integrity and uphold Enterprise's brand reputation. ## Footnote Content moderation is crucial to filter unwanted user input and block unwanted output.
37
What are the two main functions of content moderation?
* Check and filter unwanted user input such as personal information * Check and filter or block unwanted output from the LLM
38
What type of model can be used for content moderation?
Another text classification machine learning model or a data dictionary filtering predefined unwanted word list. ## Footnote This allows for customizable content moderation tailored to specific enterprise policies.
39
What is the role of AI Guardrails?
To integrate a guard rail system to the output generated by the models, checking for quality, biases, and ethical boundaries.
40
What is LLM Filters?
A Python library that filters harmful data within inbound and outbound content to the LLM engine.
41
What functionality does the FilterBlock component provide?
* Eliminates profanity * Blocks messages containing harmful content * Appends metadata to messages
42
What does the RAILS framework do?
Enforces quality checks on LLM outputs and performs pydantic-style validation of responses.
43
True or False: NeMo is an open-source toolkit designed to integrate safety measures into conversational systems.
True
44
What types of content moderation services does the SAAS-based AI platform offer?
* Identifying unwanted content in audio tracks * Detecting unpleasant communication in audio * Evaluating the quality of images * Scanning videos for prohibited scenes
45
What is the benefit of maintaining comprehensive records of AI interactions?
It demonstrates compliance with industry regulations and serves as a source of truth in disputes.
46
Fill in the blank: The process of _______ ensures storing and recording every human interaction with the AI systems.
accountability
47
What is the significance of generating a solid transactional record of interactions and content?
Establishes a foundation for accountability, transparency, and regulatory compliance.
48
What is the purpose of using a content moderation API from OpenAI?
To flag and filter unwanted content effectively.
49
What is a proactive feature of the RAILS framework?
Reasking the LLM in case the output fails the validation check.
50
What does the acronym LLM stand for?
Large Language Model
51
What is the primary advantage of using automation platforms like Zapier with NeMo?
To provide enterprise-grade AI-based automation flows.