google-research/rliable

Add support for loading data from pandas dataframe

Open

#18 opened on Jan 18, 2023

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Jupyter Notebook (49 forks)auto 404
enhancementgood first issuehelp wanted

Repository metrics

Stars
 (875 stars)
PR merge metrics
 (PR metrics pending)

Description

Right now, we only support loading data from numpy arrays. It would be nice if there was a helper function to convert a dataframe of scores to numpy arrays. Some initial code to help what this might look like:


def get_all_return_values(df):
  games = list(df['game'].unique())
  return_vals = {}
  for game in games:
    game_df = df[df['game'] == game]
    arr = game_df.groupby('wid')['normalized_score'].apply(list).values
    return_vals[game] = np.stack(arr, axis=0)
  return return_vals

def convert_to_matrix(x):
  return np.stack([x[k] for k in sorted(x.keys())], axis=1)

## Usage
# Array of shape (num_runs, num_games, num_steps)`
all_normalized_scores = convert_to_matrix(get_all_return_values(score_df))

The above code assumes we have a pandas Dataframe with keys run_number, 'gameandnormalized_score` containing scores for all steps (in a ordered manner).

Contributor guide

Research direction
Implement the helper functions get all return values and convert to matrix as outlined in the issue, ensuring they integrate smoothly with the existing rliable API. Refer to the numpy array loading code for patterns.
Tech stack
python
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
Pythonpandasnumpy
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70