Section 6 Flashcards
(26 cards)
Substages of Deploy lifecycle stage (4)
Assess readiness
Continuous monitoring
Deployment/ implementation
Adapt and govern
Readiness assessment
Opportunity discovery - identifying AI’s value-add and justifying business case
Data management - identify valuable data; establish data pipeline and governance
IT environment and security - evaluate hardware and software controls
Risk, privacy and governance including a RMF, privacy controls, governance structure, and regulatory requirements
Adoption, including change management, AI literacy, and workforce development
Deployment
move model from production environment to operational environment
Deployment environment options
Cloud, on prem, edge
Cloud environment characteristics
third-party hosted, easily scalable, latency and security risks
On prem characteristics
hosted by your organization, greater control, but larger up-front investment
Edge characteristics
hosted on local edge devices, decreased latency and better privacy, but limited by device hardware and limited computational power
Packaging
from where will the code and its dependencies by stored, configured, and deployed
“Containerization” or Container
package of software which bundles code, configuration, and dependencies to deploy software across multiple environments. Can be a data center, public cloud, or developers laptop
REST API
enables users or other applications to communicate and exchange data
Two AI access methods
REST API, embedded in application
Deployment challenges of proprietary models (5)
- Transparency requirements
- Litigation over proprietary training data
- Ownership, responsibility for outputs
- Limiting liability for high-risk applications
- Handling data breaches
Two types of third party products
Integrated, commercial off the shelf (COTS)
Visibility challenges for TPRM
○ Models may be proprietary
○ Difficulty aligning internal systems with those of third-party
○ Review vendor acceptable use policy and other documentation
Vendor agreements should cover (6)
product category, data, technical specifications, security/safety, bias and fairness, and monitoring and maintenance
End-user engagement
notify users when they are interacting with AI. When system produces decision but they are not directly interacting with the end user, the U.S. FTC requires notice
Bug bashing/bug bounties
pay a reward to ethical hackers for discovering, documenting, and disclosing newly discovered bugs or exploits
AI incident
not just an adverse impact on a user, but can also include suboptimal performance
challenger model
a new model to test against the champion or current model, and see if results can be improved
Six stages of incident response
- Preparation
- Identification
- Containment
- Eradication
- Recovery
- Lessons learned
Active learning (aka query learning)
a subfield of machine learning where algorithm can select data it learns from and request additional points to help it learn best
Entropy
the measure of unpredictability or randomness in ML dataset
Greedy algorithm
algorithms that make optimal choices to achieve immediate objectives based on available information and without regard for longer-term optimal solutions
Random forest
a supervised ML algorithm which builds and merges multiple decision trees to achieve more accurate and stable predictions. Each decision tree is made with a random subset of training data. Useful with datasets with missing values
