The Prediction Room
Metodologia e evidências
Analise a estrutura publicada, o estado das afirmações, as divulgações de incerteza, as correções e as contribuições de especialistas licenciados por trás deste documentário.
-
voiceover
The Prediction — The unfinished sentence
Start with the smallest honest description of a language model and mark what it cannot prove.
-
chronology
The Prediction — When context became parallel
Place the Transformer milestone precisely without turning architecture into mythology.
-
source_card
Training Is Not Inference — What training rewards
Show how an objective makes learning measurable—and what it leaves out.
-
graphic
Training Is Not Inference — A prompt is not a training run
Explain in-context adaptation without implying every conversation rewrites the model.
-
graphic
Training Is Not Inference — Why the biggest number is not the whole model
Replace parameter-count spectacle with a joint account of model, data, and training.
-
graphic
What Scale Measures — The smooth curve
State the regularity, then keep its metric boundary inside the documentary's main flow.
-
source_card
What Scale Measures — What the curve cannot prove
Show why extrapolating one measured relationship into every desired property is a category error.
-
archival
What Scale Measures — A breakthrough, properly bounded
Show how task, data, metric, and comparator make a breakthrough intelligible.
-
archival
Capability Without Myth — A prediction that changed a field
Examine AlphaFold through blind assessment, metric, confidence, and limits.
-
voiceover
Capability Without Myth — There is no single capability meter
Compare forms of prediction without pretending their metrics can be added into one universal rank.
-
source_card
Fluency and Uncertainty — When imitation reproduces a misconception
Use a benchmark designed around false beliefs to expose a failure hidden by fluency.
-
source_card
Fluency and Uncertainty — What the benchmark can say
Keep vintage, questions, scoring, and model set attached to the memorable percentages.
-
graphic
Fluency and Uncertainty — Three uncertainties in one answer
Separate output probabilities, knowledge limits, and the stakes of acting.
-
chronology
From Model to Agent — When prediction gets hands
Trace the move from language output to a system that can observe and act.
-
graphic
From Model to Agent — The control plane
Use NIST's functions to organize responsibility without reducing governance to a badge.
-
source_card
From Model to Agent — No certification theatre
End with continuous testing, monitoring, correction, and recourse.
-
source_card
The Material System — A dataset is an argument
Make curation visible as choices rather than a neutral mirror of the world.
-
graphic
The Material System — Average accuracy, unequal errors
Show why one score can conceal the people carrying failures.
-
map
The Material System — There is no cloud without a grid
Keep totals, local concentration, AI attribution, and scenario uncertainty separate.
-
voiceover
The Accountable Room — Prediction becomes power through a system
Synthesize without pretending one layer explains the whole outcome.
-
voiceover
The Accountable Room — Seven questions before belief
Close with a reusable evidence discipline rather than a prediction about the future.
Afirmações e evidências
-
verified · high
The AlexNet paper reports 15.3% top-five test error on ILSVRC-2012, compared with 26.2% for the second-best entry.
Atribuição: Krizhevsky, Sutskever, and Hinton (2012)
O que continua por resolver: The comparison is specific to ILSVRC-2012 and establishes neither general intelligence nor deployment reliability.
-
verified · high
The AlexNet paper describes training a deep convolutional neural network to classify 1.2 million high-resolution images into 1,000 classes.
Atribuição: Krizhevsky, Sutskever, and Hinton (2012)
O que continua por resolver: This is the paper's named task and dataset, not a definition of every later system called artificial intelligence.
-
verified · high
The Transformer paper proposed an encoder-decoder architecture based on attention mechanisms, dispensing with recurrence and convolutions in that architecture.
Atribuição: Vaswani et al. (2017)
O que continua por resolver: The paper concerns the proposed architecture and translation experiments; it did not originate attention or establish every later model behavior.
-
inference · high
An architecture diagram specifies information flow and computation; by itself it does not establish beliefs, intentions, consciousness, or a theory of mind.
Atribuição: Documentary inference from Vaswani et al. (2017) and NIST (2023)
O que continua por resolver: This separates computational description from contested mental-state attribution.
-
verified · high
The GPT-3 paper describes GPT-3 as an autoregressive language model with 175 billion parameters.
Atribuição: Brown et al. (2020)
O que continua por resolver: This identifies one 2020 model and objective; parameter count alone does not characterize current systems.
-
inference · high
For an autoregressive language model, fluent continuation is produced through conditional token prediction; token probability is not, without additional evidence, a calibrated belief that its proposition is true.
Atribuição: Documentary inference from Brown et al. (2020) and Lin et al. (2022)
O que continua por resolver: Other model families and calibration methods can expose additional uncertainty signals; this concerns what token likelihood alone licenses.
-
verified · high
The GPT-3 authors state that their pretraining objective weights every token equally and lacks a notion of what is most important to predict.
Atribuição: Brown et al. (2020)
O que continua por resolver: This is the authors' description of GPT-3's 2020 objective, not every modern training pipeline.
-
inference · high
Training changes learned parameters through optimization over examples, while inference applies learned parameters to new inputs; retrieval, tools, memory, or later fine-tuning are additional mechanisms that should be named.
Atribuição: Documentary synthesis of Vaswani et al. (2017) and Brown et al. (2020)
O que continua por resolver: Some deployed systems update memory or parameters online; those mechanisms must be identified rather than folded into ordinary inference.
-
verified · high
In the GPT-3 paper's zero-shot, one-shot, and few-shot evaluations, tasks were specified through text and the model was applied without gradient updates or fine-tuning.
Atribuição: Brown et al. (2020)
O que continua por resolver: Other products may retrieve data, call tools, update memory, or fine-tune models.
-
inference · high
Changing examples or instructions in context can change an autoregressive model's output without changing its learned weights; this is conditional computation, not proof that a training run occurred inside the conversation.
Atribuição: Documentary inference from Brown et al. (2020)
O que continua por resolver: The internal interpretation of in-context learning remains an active research question, and products may add state outside the base model.
-
verified · high
The Chinchilla study trained more than 400 language models and found, within its tested regime, that compute-optimal scaling required increasing model size and training tokens together; its 70-billion-parameter model outperformed larger comparison models at the same training-compute budget.
Atribuição: Hoffmann et al. (2022)
O que continua por resolver: The empirical result does not fix one timeless token-to-parameter rule for every architecture, dataset, or inference workload.
-
inference · high
Parameter count is one model property, not a sufficient account of data, optimization, architecture, evaluation, inference cost, or downstream behavior.
Atribuição: Documentary inference from Hoffmann et al. (2022) and Kaplan et al. (2020)
O que continua por resolver: Parameter count remains useful when surrounding training and evaluation conditions are controlled.
-
verified · high
The scaling-laws paper reports empirical power-law relationships between language-model cross-entropy loss and model size, dataset size, and training compute across the regimes it studied.
Atribuição: Kaplan et al. (2020)
O que continua por resolver: The relationships concern cross-entropy and the studied architectures, data, and compute ranges; extrapolation can fail.
-
inference · high
Smooth improvement in predictive loss does not by itself establish truthfulness, causal understanding, agency, safety, or social value; each needs a separate operational definition and evaluation.
Atribuição: Documentary inference from Kaplan et al. (2020), Lin et al. (2022), and NIST (2023)
O que continua por resolver: Predictive loss can correlate with downstream results, but the relationship must be measured rather than assumed.
-
inference · high
A benchmark result is evidence about a defined task, dataset, metric, comparator, and evaluation date; deployment claims require added evidence about context and consequences.
Atribuição: Documentary inference from Krizhevsky et al. (2012) and NIST (2023)
O que continua por resolver: Benchmarks vary in external validity; required deployment evidence depends on system and stakes.
-
verified · high
In CASP14, the AlphaFold paper reports median backbone accuracy of 0.96 angstrom r.m.s.d.95, compared with 2.8 angstrom for the next-best method, on the assessed CASP domains.
Atribuição: Jumper et al. (2021)
O que continua por resolver: The metric, domain set, confidence interval, and blind assessment define this result; not every predicted structure is correct.
-
verified · high
The AlphaFold paper reports lower accuracy when sequence alignments are shallow and weakness for proteins whose shape depends heavily on interactions with other chains.
Atribuição: Jumper et al. (2021)
O que continua por resolver: These are observed AlphaFold2 limitations; later systems and specific targets require their own evidence.
-
inference · high
AI capability is plural and task-bound: exceptional protein-structure prediction, image classification, or language continuation does not collapse into one context-free quantity called intelligence.
Atribuição: Documentary inference across three primary research programs
O que continua por resolver: General-purpose transfer can be real, but must be evaluated across explicit task distributions.
-
verified · high
TruthfulQA contains 817 questions across 38 categories designed to elicit imitative falsehoods; in the paper's evaluated model set, the best model was truthful on 58% of questions and the reported human comparison was 94%.
Atribuição: Lin, Hilton, and Evans (2022)
O que continua por resolver: The figures belong to the paper's benchmark, scoring, prompts, and model vintages; they are not current performance claims.
-
verified · high
Within the TruthfulQA paper's evaluated model set, larger models were generally less truthful, a pattern the authors connect to learning false answers from the training distribution.
Atribuição: Lin, Hilton, and Evans (2022)
O que continua por resolver: This is not a universal law that added scale always reduces truthfulness.
-
inference · medium
A responsible AI answer separates predictive uncertainty about outputs, epistemic uncertainty about adequate knowledge, and decision uncertainty about the cost of acting.
Atribuição: Documentary analytical framework grounded in Brown et al. (2020), Lin et al. (2022), and NIST (2023)
O que continua por resolver: The three-layer vocabulary is an editorial synthesis, not a standardized taxonomy in the cited sources.
-
verified · high
The ReAct paper studies prompts that interleave language-model reasoning traces with task-specific actions, allowing a system to gather information from external sources or interact with an environment.
Atribuição: Yao et al. (2023)
O que continua por resolver: ReAct is one studied design and does not establish human-like reasoning or autonomous intent.
-
inference · high
Operational agency depends on the surrounding loop: goals, tools, available data, action permissions, persistent state, and human authority to stop execution.
Atribuição: Documentary inference from Yao et al. (2023) and NIST (2023)
O que continua por resolver: Philosophical accounts differ; this concerns observable operational control in deployed systems.
-
verified · high
NIST AI RMF 1.0 organizes its Core around GOVERN, MAP, MEASURE, and MANAGE, with governance cross-cutting the other functions.
Atribuição: NIST (2023)
O que continua por resolver: AI RMF 1.0 is voluntary and under revision; the functions are not proof a particular system is safe.
-
verified · high
NIST AI RMF 1.0 says risk management should be continuous across the AI lifecycle and systems should be tested before deployment and regularly while in operation.
Atribuição: NIST (2023)
O que continua por resolver: Implementation and evidence remain the responsibility of the organization using the system.
-
inference · high
Applying a framework name is not a safety finding; responsible governance needs traceable evidence, named owners, monitoring, response, and recourse for the actual system and context.
Atribuição: Documentary inference from NIST (2023)
O que continua por resolver: Certification regimes may add audited requirements; AI RMF 1.0 itself is voluntary.
-
verified · high
Datasheets for Datasets proposes documenting a dataset's motivation, composition, collection process, recommended uses, and operating context.
Atribuição: Gebru et al. (2021)
O que continua por resolver: A datasheet improves inspectability but cannot prove collection, consent, representation, or downstream use is acceptable.
-
inference · high
Training data is not raw reality: inclusion, exclusion, labeling, filtering, deduplication, and documentation are design choices shaping what can be learned and whose experience becomes legible.
Atribuição: Documentary inference from Gebru et al. (2021)
O que continua por resolver: The influence of one data decision on behavior usually requires targeted measurement.
-
verified · high
In the Gender Shades audit of three commercial gender-classification systems, error rates reached 34.7% in its darker-skinned female subgroup, while the maximum error rate in its lighter-skinned male subgroup was 0.8%.
Atribuição: Buolamwini and Gebru (2018)
O que continua por resolver: The study's female/male labels and Fitzpatrick skin-type groupings are operational benchmark categories, not independent findings about each person's gender identity; the categories, products, benchmark, and 2018 evaluation date bound the result.
-
inference · high
Aggregate accuracy can hide sharply different error distributions; evaluation must inspect representation, subgroup errors, resulting decisions, and recourse.
Atribuição: Documentary inference from Buolamwini and Gebru (2018) and NIST (2023)
O que continua por resolver: Meaningful subgroup analyses depend on application, population, measurement quality, and legal context.
-
verified · high
The IEA estimates data centres used about 415 terawatt-hours of electricity in 2024, around 1.5% of global electricity consumption.
Atribuição: IEA (2025)
O que continua por resolver: This is for all data centres, not AI alone; global totals do not describe a facility or grid.
-
verified · high
In the IEA Base Case, global data-centre electricity consumption rises to about 945 terawatt-hours by 2030; AI is the most important growth driver alongside other digital services.
Atribuição: IEA (2025)
O que continua por resolver: This is a scenario projection, not a meter reading; adoption, efficiency, hardware, and energy bottlenecks can change it.
-
verified · high
The IEA sensitivity cases span roughly 700 to 1,700 terawatt-hours of global data-centre electricity demand in 2035.
Atribuição: IEA (2025)
O que continua por resolver: The range belongs to named cases and is not a formal confidence interval.
-
inference · high
AI-system consequences are produced jointly by model behavior, data, interfaces, permissions, institutional incentives, infrastructure, and people exposed to decisions.
Atribuição: Documentary synthesis across system, data, impact, infrastructure, and governance evidence
O que continua por resolver: The relative contribution of each layer is context-specific and must be investigated.
-
inference · high
A defensible AI claim names task, data, metric, uncertainty, action boundary, affected people, and accountable owner, then preserves monitoring and correction after deployment.
Atribuição: Documentary synthesis from Gebru et al. (2021), NIST (2023), and Lin et al. (2022)
O que continua por resolver: Specific sectors may require additional legal, scientific, or community-defined duties.
- 2026-08-29T00:00:00Z: Retained the released benchmark statement while moving its evidence reference to the newly reviewed second version of the same AlexNet source lineage and sharpening the evaluation-boundary note.
- 2026-08-29T00:00:00Z: Retained the released training-scope statement while moving its evidence reference to the newly reviewed second version of the same AlexNet source lineage and tightening the excerpt and scope boundary.
Grátis para começar, sem conta
Esta abertura para convidados é gratuita. Não é necessária conta. Você pode parar quando quiser. O progresso é salvo.