Pandas DataFrame not converting to R data.frame

[garciamikep]

Issue

Pandas DataFrame is not converting to native R data.frame, either automatically or via reticulate::r_to_py()

Steps:

• use a Python venv specified via reticulate::use_virtualenv()
• load pandas via reticulate::import()
• create a Pandas df from scratch
  • not auto-converted
• run reticulate::r_to_py() on pandas DataFrame
  • still not converted to R data.frame

Reprex

reticulate::use_virtualenv("path/to/.venv/with/pandas")

pd <- import("pandas", convert=TRUE)

# create pandas df
pd_df <- pd$DataFrame(list(a=c(1,2,3), b=c(4,5,6)))
class(pd_df) # pandas.DataFrame

# attempt to convert to R df 
r_df <- pd_df |> reticulate::r_to_py()
class(r_df) # still pandas.DataFrame

# unsurprisingly, returns pandas methods, not R data.frame columns
# I need to be able to work with r_df as an R data.frame
names(r_df$a)

reticulate::py_config()

Note: This venv has Pandas v2.3.1 installed.

python:         /Users/mike.garcia/Desktop/pd-reticulate/.venv/bin/python
libpython:      /Users/mike.garcia/.pyenv/versions/3.11.9/lib/libpython3.11.dylib
pythonhome:     /Users/mike.garcia/Desktop/pd-reticulate/.venv:/Users/mike.garcia/Desktop/pd-reticulate/.venv
version:        3.11.9 (main, Sep  4 2024, 09:25:45) [Clang 15.0.0 (clang-1500.3.9.4)]
numpy:          /Users/mike.garcia/Desktop/pd-reticulate/.venv/lib/python3.11/site-packages/numpy
numpy_version:  2.4.1
pandas:         /Users/mike.garcia/Desktop/pd-reticulate/.venv/lib/python3.11/site-packages/pandas

NOTE: Python version was forced by use_python() function

sessionInfo()

R version 4.5.2 (2025-10-31)
Platform: aarch64-apple-darwin20
Running under: macOS Tahoe 26.2

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] reticulate_1.44.1   data.table_1.18.2.1 devtools_2.4.6     
[4] usethis_3.2.1      

loaded via a namespace (and not attached):
 [1] R6_2.6.1          fastmap_1.2.0     Matrix_1.7-4      lattice_0.22-7   
 [5] magrittr_2.0.4    remotes_2.5.0     cachem_1.1.0      glue_1.8.0       
 [9] memoise_2.0.1     png_0.1-8         lifecycle_1.0.5   cli_3.6.5        
[13] sessioninfo_1.2.3 grid_4.5.2        vctrs_0.7.1       renv_0.15.0      
[17] compiler_4.5.2    pkgload_1.4.1     purrr_1.2.1       tools_4.5.2      
[21] pkgbuild_1.4.8    ellipsis_0.3.2    Rcpp_1.1.1        jsonlite_2.0.0   
[25] rlang_1.1.7       fs_1.6.6     

[t-kalinowski]

Did you perhaps mean to call py_to_r() instead of r_to_py()?

library(reticulate)
py_require("pandas<3")

pd <- import("pandas", convert=FALSE)

# create pandas df
pd_df <- pd$DataFrame(list(a=c(1,2,3), b=c(4,5,6)))
class(pd_df) # pandas.DataFrame
#> [1] "pandas.core.frame.DataFrame"        "pandas.core.generic.NDFrame"       
#> [3] "pandas.core.base.PandasObject"      "pandas.core.accessor.DirNamesMixin"
#> [5] "pandas.core.indexing.IndexingMixin" "pandas.core.arraylike.OpsMixin"    
#> [7] "python.builtin.object"

# attempt to convert to R df 
r_df <- py_to_r(pd_df)
class(r_df) # still pandas.DataFrame
#> [1] "data.frame"

^{Created on 2026-01-28 with reprex v2.1.1}

[garciamikep]

Ah, sorry that was a typo. Yes, I meant to call py_to_r(). Interestingly, it doesn't seem to matter – I get the same result (pandas.DataFrame) in either case:

r_df <- pd_df |> reticulate::py_to_r()
class(r_df) 
# [1] "pandas.DataFrame" "pandas.core.generic.NDFrame" "pandas.core.base.PandasObject" "pandas.core.accessor.DirNamesMixin"
# [5] "pandas.core.indexing.IndexingMixin" "pandas.core.arraylike.OpsMixin" "python.builtin.object" 

# w/ typo
r_df_wrong <- pd_df |> reticulate::r_to_py()
class(r_df_wrong) 
# [1] "pandas.DataFrame" "pandas.core.generic.NDFrame" "pandas.core.base.PandasObject" "pandas.core.accessor.DirNamesMixin"
# [5] "pandas.core.indexing.IndexingMixin" "pandas.core.arraylike.OpsMixin" "python.builtin.object"   

Also noting that I'm unable to successfully import pandas at all if I use py_require("pandas<3") instead of use_virtualenv().

[t-kalinowski]

Would you be able to share a MRE (including instructions on how to recreate the venv)? My best guess is that you have pandas version >=3 in the .venv.

If py_require() didn't work, you probably have a custom self-managed venv earlier in the order of discovery. You can force reticulate to ignore other entries in the order of discovery and use an ephemeral venv by setting Sys.setenv(RETICULATE_PYTHON = "managed")

[garciamikep]

Sure, here's an MRE including details on environment setup. You are correct – I have pandas 3.0.0. Does reticulate object conversion only work for pandas < 3?

My full use case involves calling a custom Python package for data collection from R, then manipulating the resulting data (as a data.frame). If the data collection package requires pandas >= 3, then that's what will be in the environment I need to use (not sure an ephemeral venv would work here?)

MRE

Python environment

cd path/to/project

python -m venv .venv

.venv/bin/pip3 install pandas

# confirm versions
.venv/bin/python3 --version #  3.11.9
.venv/bin/python3 -c "import pandas; print(pandas.__version__)" #  3.0.0

R session

R 4.5.2, using fresh renv

library(reticulate)

venv_path <- "path/to/project/.venv"
use_virtualenv(venv_path)

pd <- import("pandas", convert=TRUE)

py_pkg <- py_list_packages()
py_pkg[py_pkg$package == 'pandas', ] # v3.0.0

# create pandas df
pd_df <- pd$DataFrame(list(a=c(1,2,3), b=c(4,5,6)))
class(pd_df) # pandas.DataFrame

# attempt to convert to R df 
r_df <- pd_df |> reticulate::py_to_r()
class(r_df) 

[eddelbuettel]

Yes. See #1874 filed yesterday.

[garciamikep]

Thanks – turns out my internal dependency is compatible with pandas < 3, so reconfiguring my python environment will work for now.

Appreciate the help!

[eddelbuettel]

@t-kalinowski had also mentioned in earlier posts but maybe not as "loudly" as needed. We are all busy, and under permanent attention deficit-disorder....