Update copy (#39)

about.py

@@ -5,52 +5,52 @@ def build_page():
     gr.Markdown(
         """
 ## About AstaBench
-AstaBench is a best-in-class AI agents evaluation framework to measure scientific research abilities. AstaBench provides a challenging new test for AI agents: the first benchmark challenge that evaluates agents’ scientific abilities on a broad spectrum of research skills, including literature understanding, data analysis, planning, tool use, coding, and search.
+AstaBench is a novel AI agents evaluation framework, providing a challenging new test for AI agents: the first benchmark challenge that evaluates agents’ scientific abilities on a broad spectrum of research skills, including literature understanding, data analysis, planning, tool use, coding, and search. Asta’s set of standard tools makes it easy to build general-purpose science agents and to compare their performance in an apples-to-apples manner.
 
 
-**Why AstaBench?**
-Newer benchmarks may test agentic AI and isolated aspects of scientific reasoning, but none rigorously measure agentic AI or capture the full range of skills research demands. Agents can appear effective by simply retrying tasks—often at high computational cost and with inconsistent results. Scientific AI needs evaluations that reflect the real complexity of research.
+## Why AstaBench?
+Most current benchmarks test agentic AI and isolated aspects of scientific reasoning, but rarely evaluate AI agentic behavior rigorously or capture the full skill set scientific research requires. Agents can appear effective despite inconsistent results and high compute use, often outperforming others by consuming more resources. Advancing scientific AI requires evaluations that emphasize reproducibility, efficiency, and the real complexity of research.
 
-AstaBench fills that gap: a suite of open benchmarks for evaluating scientific AI assistants on core scientific tasks that require novel reasoning. AstaBench helps scientists identify which agents best support their needs through task-relevant leaderboards, while giving AI developers a standard execution environment and standard tools to test the scientific reasoning capabilities of their agents compared to well-known baselines from the literature, including both open and closed LLM foundation models.
+AstaBench fills this gap: an agents evaluation framework and suite of open benchmarks for evaluating scientific AI assistants on core scientific tasks that require novel reasoning. AstaBench helps scientists identify which agents best support their needs through task-relevant leaderboards, while giving AI developers a standard execution environment and tools to test the scientific reasoning capabilities of their agents compared to well-known baselines from the literature, including both open and closed LLM foundation models.
 
 
-**What Does AstaBench Include?**
-The suite includes over 8,000 tasks across 11 benchmarks, organized into four core categories:
-- Literature Understanding
-- Code & Execution
-- Data Analysis
-- End-to-End Discovery
+## What Does AstaBench Include?
+AstaBench includes a rigorous agents evaluation framework and a suite of benchmarks consisting of over 2,400 problems across 11 benchmarks, organized into four core categories:
+Literature Understanding
+Code & Execution
+Data Analysis
+End-to-End Discovery
+Plus: a large suite of integrated agents and leaderboards with results from extensive evaluation of agents and models.
 
 🔍 Learn more in the AstaBench technical blog post
 
 
-**Understanding the Leaderboards**
-The AstaBench Main Leaderboard provides a high-level view of overall agent performance and efficiency:
+## Understanding the Leaderboards
+The AstaBench Overall Leaderboard provides a high-level view of overall agent performance and efficiency:
 - Overall score: A macro-average of the four category-level averages (equal weighting)
 - Overall cost: Average cost per task, aggregated only across benchmarks with reported cost
 
 Each category leaderboard provides:
-- Average score and cost for that category
+- Average score and cost for that category (macro-averaged across the benchmarks in the category)
 - A breakdown by individual benchmarks
 
 
-**Scoring & Aggregation**
-AstaBench encourages broad, honest evaluation. Here's how we handle scoring, cost, and partial results:
+## Scoring & Aggregation
+AstaBench encourages careful, transparent evaluation. Here's how we handle scoring, cost, and partial results:
 
-_Scores_
-- Each benchmark returns an average score based on per-task accuracy
-- Skipped benchmarks receive a score of 0.00
+**Scores**
+- Each benchmark returns an average score based on per-problem scores
 - All scores are aggregated upward using macro-averaging
 - Partial completions are included (even with poor performance)
 
-_Cost_
-- Costs are reported in USD per task, based on values at the time of submission
+**Cost**
+- Costs are reported in USD per task.
 - Benchmarks without cost data are excluded from cost averages
-- In scatter plots, agents without cost are plotted far right and clearly marked
-Note: Cost values reflect pricing and infrastructure conditions at the time of each submission. We recognize that compute costs may change over time, and are actively working on methods to normalize cost data across submissions for fairer longitudinal comparisons.
+- In scatter plots, agents without cost are plotted to the far right and clearly marked. 
 
+Note: Cost values reflect pricing and infrastructure conditions at the time of each submission. We recognize that compute costs may change over time, and are actively working on methods to normalize cost data across submissions for fairer longitudinal comparisons.
 
-_Coverage_
+**Coverage**
 - Main leaderboard: category coverage (X/4)
 - Category view: benchmark coverage (X/Y)
 - Incomplete coverage is flagged visually
@@ -58,7 +58,7 @@ _Coverage_
 These design choices ensure fair comparison while penalizing cherry-picking and omissions.
 
 
-**Learn More**
+## Learn More
 - AstaBench technical blog post
 - FAQ and submission guide
 """, elem_id="about-content"

category_page_builder.py

@@ -13,7 +13,7 @@ def build_category_page(CATEGORY_NAME, PAGE_DESCRIPTION):
 
         with gr.Column(elem_id="test_nav_container", visible=True) as test_nav_container:
             create_sub_navigation_bar(test_tag_map, CATEGORY_NAME)
-        gr.Markdown(f"## Astabench{CATEGORY_NAME} Leaderboard")
+        gr.Markdown(f"## Astabench{CATEGORY_NAME} Leaderboard (Aggregate)")
         gr.Markdown(PAGE_DESCRIPTION, elem_id="category-intro")
         # --- This page now has two main sections: Validation and Test ---
         with gr.Tabs():

content.py

@@ -38,57 +38,31 @@ Agents names that are green are Pareto optimal, meaning they achieve the best pe
 """
 LIT_DESCRIPTION = """
 The **Literature Understanding** category evaluates how well agents comprehend and interact with scientific literature—testing their ability to find research papers, assess citation quality, extract information from text, and more.
+<br><br>
+The scores shown below reflect performance aggregated across five distinct benchmarks, each targeting a different aspect of literature-based reasoning. 
+<br><br>
+For detailed results, use the links above to explore individual benchmarks.
 <br>
-The scores shown below reflect performance aggregated across five distinct benchmarks, each targeting a different aspect of literature-based reasoning:
-<br>
-- PaperFinding Bench – PLACEHOLDER DESCRIPTION
-<br>
-- ScholarQA Bench2 – PLACEHOLDER DESCRIPTION
-<br>
-- LitQA2-FT – PLACEHOLDER DESCRIPTION
-<br>
-- ArxivDIGES Tables-Clean – PLACEHOLDER DESCRIPTION
-<br>
-<br>
-Together, these tasks form a comprehensive evaluation of an agent’s ability to navigate, understand, and reason over scientific publications
 """
 CODE_EXECUTION_DESCRIPTION = """
-The **Code & Execution** category in AstaBench includes tasks that evaluate an agent’s ability to write, modify, and run code in realistic research scenarios. Unlike literature tasks—which can sometimes be solved by a language model alone—these problems often require the agent to interact with tools: reading input files, executing code, and writing outputs to specific files in the required format.
-<br>
-<br>
-The scores in this category are aggregated from three distinct benchmarks, each targeting different facets of scientific coding and execution:
-<br>
-- CORE-Bench-Hard – PLACEHOLDER DESCRIPTION
-<br>
-- DS-1000 – PLACEHOLDER DESCRIPTION
-<br>
-- SUPER-Expert – PLACEHOLDER DESCRIPTION
+The **Code & Execution** category in AstaBench includes tasks that evaluate an agent’s ability to write, modify, and run code in realistic research scenarios. Unlike literature tasks—which only require read-only tools and can sometimes even be solved by a language model alone—these problems often require the agent to manipulate a machine environment with tools: reading input files, executing code, and writing outputs to specific files in the required format.
+<br><br>
+The scores in this category are aggregated from three distinct benchmarks, each targeting different facets of scientific coding and execution. Together, these benchmarks evaluate whether an agent can function as a hands-on scientific assistant—not just by reasoning about code, but by running it in real-world contexts.
+<br><br>
+For detailed results, use the links above to explore individual benchmark pages.
 <br>
-<br>
-Together, these benchmarks evaluate whether an agent can function as a hands-on scientific assistant—not just by reasoning about code, but by running it in real-world contexts.
 """
 DATA_ANALYSIS_DESCRIPTION = """
-The **Data Analysis** category evaluates agents on their ability to analyze structured datasets and generate meaningful scientific hypotheses. It currently includes a single benchmark:
-<br>
- - DiscoveryBench
-<br>
-so the category-level scores are the same as the benchmark-level results.
-<br>
-<br>
+The **Data Analysis** category evaluates agents on their ability to analyze structured datasets and generate meaningful scientific hypotheses. It currently includes a single benchmark, DiscoveryBench, so the category-level scores are the same as the benchmark-level results.
+<br><br>
 As additional benchmarks are added in the future, this category will expand to cover a broader range of data-driven reasoning tasks across scientific domains.
+<br>
 """
 DISCOVERY_DESCRIPTION = """
-The **End-to-End Discovery** category tests whether agents can carry out a complete scientific workflow—from hypothesis generation and experiment design to code execution, analysis, and report writing. These tasks require agents to integrate multiple capabilities, producing not just answers but full research artifacts.
-<br>
-<br>
-Scores in this category are aggregated from two benchmarks:
-<br>
-- E2E-Bench – PLACEHOLDER DESCRIPTION
-<br>
-- E2E-Bench-Hard – PLACEHOLDER DESCRIPTION
-<br>
+The **End-to-End Discovery** category tests whether agents can carry out a complete scientific workflow, from task description to experiment design, code execution, results  analysis, and report writing. These tasks require agents to integrate multiple capabilities, producing not just answers but full research artifacts.
+<br><br>
+Scores in this category are aggregated from two benchmarks, providing the first standardized way to evaluate automated scientific discovery (ASD) agents across all stages of the research process. Use the links above to explore individual benchmark pages.
 <br>
-This category provides the first standardized way to evaluate automated scientific discovery (ASD) agents across all stages of the research process.
 """
 
 CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"