issue regarding running k2sc -- having trouble reproducing the k2sc team's results

[bethbiller]

We've been using k2sc v 1.0 (downloaded from this repository) with george v 0.3.1 to detrend the lightcurves and remove telescope systematics. Unfortunately, we're having trouble replicating the lightcurves provided by your team here:

https://archive.stsci.edu/prepds/k2sc/

I've attached an example -- this is k2 id 210327027:

also known as 2MASS 0355+11, an L3-L6 brown dwarf observed during campaign 4. In the attached plot, the top panel is the raw PDCSAP flux, the middle panel is our attempt to run k2sc on the PDCSAP flux (clearly something off), and the bottom panel is the result from the lightcurves provided by your team (clearly detrended). All lightcurves have been normalised by the median value, and then had 1.0 subtracted from them (so units should be fractional variation from the median).

We installed k2sc following the directions on this github repository, and with the command line:

k2sc ktwo210327027-c04_llc.fits

where ktwo210327027-c04_llc.fits is the lightcurve file downloaded directly from the MAST archive.

I've put both the input and output (EPIC_210327027_mast.fits) files up on Dropbox, along with the script used to plot the lightcurves (plot_lightcurves.py):

https://www.dropbox.com/sh/u3kgy2j7f1jh4i2/AABMSHSti1d2h6k06nNyLf6ka?dl=0

Any insight you might have into what is going on (and what we might have done wrong) would be greatly appreciated! Thanks in advance!

[benjaminpope]

Hi Beth,

Just to update you, I have been able to reproduce your bug but haven't been able to fix it. Running it either from the command line or from the lightkurve API gives the same bug. I will keep you updated.

@hpparvi do you have any ideas?

All the best,

Ben

[hpparvi]

Hi Beth,

Sorry, I'm on holidays and haven't been checking my mail that frequently. I'll try to find some time to check this within the next day or two.

Cheers,
Hannu

[bethbiller]

Hi Ben and Hannu --

Just checking in if there had been any resolution as to what is causing this bug? Thanks in advance!

Best Wishes,
Beth

[benjaminpope]

Unfortunately not yet!

…

On Mon, Mar 2, 2020 at 5:05 AM bethbiller ***@***.***> wrote: Hi Ben and Hannu -- Just checking in if there had been any resolution as to what is causing this bug? Thanks in advance! Best Wishes, Beth — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DABN6YFPKFHPEONQSZPILZM3RFOAG5A5CNFSM4KWO7ZCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENOWF4I-23issuecomment-2D593322737&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=zG_oIXjhxT5xnDPHZiCpj0xiQKOvHsAbjwib7X8AotE&s=a2jyvEF3BctUG8NYaVkZ-XwoqR967fl3aDVEAYA_ook&e=>, or unsubscribe <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFLSYRVQQWVLF6RCGWDRFOAG5ANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=zG_oIXjhxT5xnDPHZiCpj0xiQKOvHsAbjwib7X8AotE&s=tOjxM10w8qW6oVpJtaLc6ktksU--BC9wfxXUx_DYZ3c&e=> .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io

[bethbiller]

Hi Ben and Hannu --

Just checking in on this. I have a new research student starting this semester who is interested in running k2sc on K2 brown dwarf lightcurves, so wanted to check if you thought it was possible for this bug to be resolved within the next month or two? If not, I will suggest a different research project for him.

Thanks!

Best Wishes,
Beth

[benjaminpope]

Hi Beth, I'm sorry, I have just been completely slammed and k2sc has been a low priority for me. Hannu, have you had a look at this? I apologize for being useless, Beth - if this is a research project that you really want to do, I can prioritize this. All the best, Ben

…

On Thu, Sep 17, 2020 at 8:40 PM bethbiller ***@***.***> wrote: Hi Ben and Hannu -- Just checking in on this. I have a new research student starting this semester who is interested in running k2sc on K2 brown dwarf lightcurves, so wanted to check if you thought it was possible for this bug to be resolved within the next month or two? If not, I will suggest a different research project for him. Thanks! Best Wishes, Beth — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-23issuecomment-2D694148929&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=nOziImNtBGzDhdY6YnHmc1vC0mDrqQgbvVpjMfjz1p8&s=aD05M22OBDffntjyFae8w_APUjrlWJvvWcuRMV3-Nu0&e=>, or unsubscribe <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFKSZYBRJVQRCYCPZ2TSGHRSRANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=nOziImNtBGzDhdY6YnHmc1vC0mDrqQgbvVpjMfjz1p8&s=Ch4gdrMztrk4c1Q_uzONjhm7D4o26n8CvnFEJr0R2HE&e=> .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io

[bethbiller]

Hi Ben and Hannu --

So my new student has started and is in fact very keen on working with k2sc on K2 brown dwarf lightcurves.
We've tried some of the other detrending software available (Everest, k2sff, etc.) and honestly, only k2sc does a good job detrending the light curves for these faint objects! K2sc will be quite pivotal for this project -- considering this, we're also happy to include the k2sc team on any eventual publication from this work. Thanks!

Best Wishes,
Beth

[saigrain]

Hi Beth Glad to hear it! I’m sure you’ve seen it already, but just in case, this paper used K2SC for K2 Brown Dwarf LCs too: https://ui.adsabs.harvard.edu/abs/2018ApJ...859..153S/abstract In that case the K2SC analysis was more a sanity check – I ran t for them after they had done their own analysis, to confirm that it gave consistent periods (which it did). Cheers Suzanne

…

-- Professor of Astrophysics University of Oxford Fellow of All Souls College Suzanne.Aigrain@physics.ox.ac.uk +44 1865 273 339 / +44 7549 939 899 From: bethbiller <notifications@github.com> Reply to: OxES/k2sc <reply@reply.github.com> Date: Thursday, 24 September 2020 at 10:37 To: OxES/k2sc <k2sc@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [OxES/k2sc] issue regarding running k2sc -- having trouble reproducing the k2sc team's results (#24) Hi Ben and Hannu -- So my new student has started and is in fact very keen on working with k2sc on K2 brown dwarf lightcurves. We've tried some of the other detrending software available (Everest, k2sff, etc.) and honestly, only k2sc does a good job detrending the light curves for these faint objects! K2sc will be quite pivotal for this project -- considering this, we're also happy to include the k2sc team on any eventual publication from this work. Thanks! Best Wishes, Beth — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#24 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAAKAOICL47CYTJUG4FVLD3SHMHJZANCNFSM4KWO7ZCA>.

[benjaminpope]

Hi @bethbiller,

Sorry to be so lax about this. I am a little overwhelmed at the moment. @hpparvi have you plans to look at this? Or else I can.

Beth - what timescale do you need the fixes by? I have a deadline mid next week after which I may be able to get to this.

All the best,

Ben

[hpparvi]

Hi,

Sorry, I've been busy also for the last months, but will try to find time to fix the issue during the next two weeks.

Cheers,
Hannu

[bethbiller]

Hi everyone --

A fix in the next month or so would be great.

Best Wishes,
Beth

[benjaminpope]

Hi @hpparvi,

Ok so I'm looking at this and something is clearly wrong. Are the light curves labelled correctly or normalized in some unusual way? I can't actually remember - it was four or five years ago - but just overplotting trtime and trposi on @bethbiller's exact plots, it's clear that the corrections are much smaller than the differences.

Ben

[benjaminpope]

k2sc's standalone version appears to reproduce fairly well Figure 3 from the original paper, correcting EPIC 211748059: https://arxiv.org/abs/1603.09167

[bethbiller]

Hi everyone -- Just checking in on this. While the current standalone version doesn't work for the campaign 4 observation k2 id 210327027, there clearly was an earlier version of k2sc that did provide sensible corrections -- the lightcurve available here: https://archive.stsci.edu/prepds/k2sc/ looks well-corrected. Any chance of accessing that earlier version of k2sc for the additional K2 brown dwarf lightcurves we are looking at?

[benjaminpope]

Hannu - which version of k2sc did the HLSPs use? What made the difference here, I wonder?

…

On Tue, Nov 10, 2020 at 10:25 PM bethbiller ***@***.***> wrote: Hi everyone -- Just checking in on this. While the current standalone version doesn't work for the campaign 4 observation k2 id 210327027, there clearly was an earlier version of k2sc that did provide sensible corrections -- the lightcurve available here: https://archive.stsci.edu/prepds/k2sc/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__archive.stsci.edu_prepds_k2sc_&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=U5NDHDengck4TRhaZJ_B4jchJIUruzNl56yEQhj9HD0&s=n-8_nd7yUuSGOwQ2v8aXMIm9rC--0D3yinigNpKj97U&e=> looks well-corrected. Any chance of accessing that earlier version of k2sc for the additional K2 brown dwarf lightcurves we are looking at? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-23issuecomment-2D724641467&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=U5NDHDengck4TRhaZJ_B4jchJIUruzNl56yEQhj9HD0&s=9bsz_GyzSP_NYUKdFhyZs4wJWUBJRPdIaXlqFRfu9Rc&e=>, or unsubscribe <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFKX7T6P3FMN2FAR5P3SPEPIXANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=U5NDHDengck4TRhaZJ_B4jchJIUruzNl56yEQhj9HD0&s=xDez3naCFHhKO1agv2wfKe09YwsSuSr3IdREUwwg93s&e=> .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io