update to master and move dictionary item in NEWS to version 3.0-1

CONTRIBUTING.md

@@ -28,7 +28,7 @@ The sections below outline the steps in each case.
 3. if needed, fork the repository to your own Github profile and create your own feature branch off of the latest master commit. While working on your feature branch, make sure to stay up to date with the master branch by pulling in changes, possibly from the 'upstream' repository (follow the instructions [here](https://help.github.com/articles/configuring-a-remote-for-a-fork/) and [here](https://help.github.com/articles/syncing-a-fork/));
 4. make sure the existing tests still work by running the test suite from RStudio;
 5. add your own tests (if necessary);
-6. update or expand the documentation;
+6. update or expand the documentation, see [package documentation guidelines](#package-documentation);
 7. make sure the release notes in `inst/NEWS.Rd` are updated;
 8. add your name to the contributors lists in the `DESCRIPTION` and `CITATION.cff` files;
 9. push your feature branch to (your fork of) the GGIR repository on GitHub;
@@ -51,6 +51,114 @@ Some general guidelines that we try to adhere to:
 If you are a first time contributor, don't worry about coding style too much.
 We will help you get things in shape.
 
+### Package documentation
+
+We currently have three sources for documenting the package:
+
+- The reference manual, including package basic information and the functions documentation files.
+- The package vignettes.
+- The github.io website (built with the `pkgdown` package).
+
+#### Reference manual
+
+The reference manual gets the information from the .Rd documents within the man
+folder in the package repository. Therefore, updating the information in those
+files will automatically update the reference manual. Note that most of the GGIR
+functions are not intended for direct interaction with the user, as such, the
+documentation of most arguments is centralized in the details section of man/GGIR.Rd. 
+If you for example want to add an extra parameter to `params_247` then this should
+be documented there. Further, you should not forget to include the new argument in
+functions \link{load_params} and \link{check_params}.
+
+#### Package vignettes
+
+The folder vignettes in the GGIR repository contains the .Rmd files. The .Rmd files
+that do not start with the word 'chapter' are used for the traditional package vignettes
+as hosted by CRAN. Use these files to edit an existing vignette, or use the structure
+of any of the vignettes to build up a new one. All .Rmd files which name 
+starts with the word 'chapter' are ignored. These chapter-vignettes are used only for
+the github.io website (see [next section](#github.io-website)).
+
+**To create a new vignette for CRAN**
+
+To create a new package vignette for CRAN, please use `usethis::use_vignette()` and
+make sure the name of the vignette file does not start by "chapter". For example,
+if you want to create a new vignette on sleep for CRAN, you may do the following:
+
+```
+usethis::use_vignette(name = "sleep", 
+                      title = "How to analyse your sleep data in GGIR")
+```
+This would create a new "sleep.Rmd" file within the vignettes folder in the GGIR
+repository. Then you can edit this file to build up the vignette.
+
+**To remove a vignette from CRAN**
+
+There are two ways to remove a vignette from CRAN:
+
+1. Removing the Rmd file corresponding to the vignette in the vignettes folder,
+note that the file and the information will be lost.
+2. Adding the path to the vignette in the .Rbuildignore file available in the
+GGIR repository. For example, to remove the GGIRParameters vignette from CRAN, you
+can add:
+
+```
+^vignettes/GGIRParameters.Rmd
+
+```
+
+#### github.io website
+
+For updating or adding information to the github.io website, we need to use
+[the pkgdown configuration file](_pkgdown.yml) that can be found in the repositories 
+root directory, as well as with the chapter vignettes discussed above.
+
+**To edit information in an existing chapter**
+
+1. Open the vignette corresponding to the chapter you wish to edit (see the _pkgdown.yml) 
+file for the chapter and its vignette path (href).
+2. Make your changes in the vignette.
+3. Run the `pkgdown::build_site()` function.
+
+**To add a new chapter**
+
+1. Create a Rmd file for the vignette via `usethis::use_vignette()` and make sure the 
+name of the vignette starts by "chapter", for example: 
+
+```
+usethis::use_vignette(name = "chapterSleep", 
+                      title = "10. How to analyse your sleep data in GGIR")
+```
+
+2. Open the [_pkgdown.yml](_pkgdown.yml) file and fill up the name and reference of the
+new chapter under menu. Make sure to follow the coding and structure of the rest of chapters.
+3. Run the `pkgdown::build_site()` function.
+
+**To remove a chapter**
+
+1. Remove the lines corresponding to the chapter in the [_pkgdown.yml](_pkgdown.yml) file
+in line 42 onwards.
+2. Optionally you may remove the Rmd file corresponding to the chapter, but only by doing the
+step 1, the chapter will not appear in the github.io website.
+3. Run the `pkgdown::build_site()` function.
+
+**To edit the name of a chapter**
+
+Chapter names are defined twice, in the [_pkgdown.yml](_pkgdown.yml) file and in the
+vignette file itself. You need to make sure both titles match as the first will be 
+used in the drop-down list in the github.io website and the other in the specific
+page for the chapter. 
+
+#### Adding the changes to the master branch
+
+The last step would be committing and pushing your changes to github and making a
+pull request as with any other contribution to the package. Note that, running the 
+`pkgdown::build_site()` function will edit the files within the docs folder, and 
+probably add some new files. This only applies when editing information from the 
+github.io website. It is important that these changes to the files in the
+docs folder are also part of the pull requests, as otherwise the website would not 
+be updated.
+
 ## New release
 
-GGIR follows the [release cycle process described in this document](RELEASE_CYCLE.md).
+GGIR follows the [release cycle process described in this document](RELEASE_CYCLE.md).

DESCRIPTION

@@ -1,8 +1,8 @@
 Package: GGIR
 Type: Package
 Title: Raw Accelerometer Data Analysis
-Version: 2.10-4
-Date: 2023-10-05
+Version: 3.0-0
+Date: 2023-10-16
 Authors@R: c(person("Vincent T","van Hees",role=c("aut","cre"),
                     email="[email protected]"),
              person("Jairo H","Migueles",role="aut",

NEWS.md

@@ -1,4 +1,10 @@
-# CHANGES IN GGIR VERSION 2.10-5
+# CHANGES IN GGIR VERSION 3.0-1
+
+- Part 5: New function to provide definition of the variable names in part 5 reports #938
+
+# CHANGES IN GGIR VERSION 3.0-0
+
+- Part 1 and 2: Change default value for nonwear_approach to "2023" which labels the entire window as nonwear if conditions are met. This instead of only the middle 15 minutes as is the case for "2013" setting. Further, setting "2023" now uses a 5 Hertz version of the signals for non-wear detection, for clipping detection the code uses the original signal.
 
 - Part 2: Move cosinor analysis code to its own function in order to ease re-using it in both part 2 and part 6.
 
@@ -7,7 +13,9 @@
 - Part 2: Arguments hrs.del.start and hrs.del.end when combined with strategy = 3 and strategy = 5 now count
 relative to start and end of the most active time window as identified.  #905
 
-- Part 5: New function to provide definition of the variable names in part 5 reports #938
+- Part 5: Change default for segmentDAYSPTcrit.part5 from c(0,0) to c(0, 0.9) and now prohibiting the use of c(0, 0) as it gives biased estimates. We knew this, but some users started using the default without attempting to understand it, by which it seems necessary to force a sensible default. #940
+
+- Part 5: Added optioned "OO" to argument timewindow, which defines windows from sleep Onset to sleep Onset #931
 
 # CHANGES IN GGIR VERSION 2.10-4
 
@@ -27,6 +35,8 @@ relative to start and end of the most active time window as identified.  #905
 
 - Fix recently introduced bug where GGIR environment was not exported to cluster in GGIR part 1, 2, 3, and 5 #910
 
+- Add contributing guidelines for github.io documentation #923
+
 # CHANGES IN GGIR VERSION 2.10-3
 
 - Part 1: Fixed minor bug in ismovisens that failed when datadir started with "./" #897

R/GGIR.R

@@ -152,8 +152,6 @@ GGIR = function(mode = 1:5, datadir = c(), outputdir = c(),
   }
 
   if (length(myfun) != 0) { # Run check on myfun object, if provided
-    warning("\nAre you using GGIR as online service to others? If yes, then make sure you prohibit the",
-            " user from specifying argument myfun as this poses a security risk.", call. = FALSE)
     check_myfun(myfun, params_general[["windowsizes"]])
   }
 

R/check_params.R

@@ -397,9 +397,22 @@ check_params = function(params_sleep = c(), params_metrics = c(),
                   "fraction of the day between zero and one, please change."), 
            call. = FALSE)
     }
+    if (length(params_cleaning[["segmentDAYSPTcrit.part5"]]) != 2) {
+      stop("\nArgument segmentDAYSPTcrit.part5 is expected to be a numeric vector of length 2", call. = FALSE)
+    }
+    if (sum(params_cleaning[["segmentDAYSPTcrit.part5"]]) < 0.5 |
+        0 %in% params_cleaning[["segmentDAYSPTcrit.part5"]] == FALSE) {
+
+      stop(paste0("\nIf you used argument segmentDAYSPTcrit.part5 then make sure",
+                  " it includes one zero",
+                  " and one value of at least 0.5, see documentation for",
+                  " argument segmentDAYSPTcrit.part5. If you do not use",
+                  " argument segmentDAYSPTcrit.part5",
+                  " then delete it from your config.csv file (in your output folder)",
+                  " or delete the config.csv file itself."), call. = FALSE)
+    }
   }
 
-
   invisible(list(params_sleep = params_sleep,
                  params_metrics = params_metrics,
                  params_rawdata = params_rawdata,

R/detect_nonwear_clipping.R

@@ -11,6 +11,11 @@ detect_nonwear_clipping = function(data = c(), windowsizes = c(5, 900, 3600), sf
   CWav = NWav = rep(0, nmin)
   crit = ((window/window2)/2) + 1
 
+  if (is.numeric(data[,1]) == FALSE) {
+    data[,1] = as.numeric(data[,1])
+    data[,2] = as.numeric(data[,2])
+    data[,3] = as.numeric(data[,3])
+  }
   if (nonwear_approach %in% c("2013", "2023")) {
     # define windows to check:
     for (h in 1:nmin) { #number of windows
@@ -26,8 +31,8 @@ detect_nonwear_clipping = function(data = c(), windowsizes = c(5, 900, 3600), sf
           hoc1 = 1
           hoc2 = window
         } else if (h >= (nmin - crit)) {
-          hoc1 = (nmin - crit)*window2
-          hoc2 = nmin*window2 #end of data
+          hoc1 = (nmin - crit) * window2
+          hoc2 = nmin * window2 #end of data
         } else if (h > crit & h < (nmin - crit)) {
           hoc1 = (((h - 1) * window2) + window2 * 0.5 ) - window * 0.5
           hoc2 = (((h - 1) * window2) + window2 * 0.5 ) + window * 0.5
@@ -38,7 +43,7 @@ detect_nonwear_clipping = function(data = c(), windowsizes = c(5, 900, 3600), sf
         NWflag = h:(h + window/window2 - 1)
         if (NWflag[length(NWflag)] > nmin) NWflag = NWflag[-which(NWflag > nmin)]
         # window to check (not aggregated values)
-        hoc1 = h*window2 - window2 + 1
+        hoc1 = h * window2 - window2 + 1
         hoc2 = hoc1 + window - 1
         if (hoc2 > nrow(data)) {
           hoc2 = nrow(data)
@@ -53,35 +58,38 @@ detect_nonwear_clipping = function(data = c(), windowsizes = c(5, 900, 3600), sf
       }
       for (jj in  1:3) {
         # Clipping
-        CW[h,jj] = length(which(abs(as.numeric(data[(1 + cliphoc1):cliphoc2,jj])) > clipthres))
-        if (length(which(abs(as.numeric(data[(1 + cliphoc1):cliphoc2,jj])) > clipthres*1.5)) > 0) {
-          CW[h,jj] = window2 # If there is a a value that is more than 150% the dynamic range then ignore entire block.
+        aboveThreshold = which(abs(data[(1 + cliphoc1):cliphoc2, jj]) > clipthres)
+        CW[h, jj] = length(aboveThreshold)
+        if (length(aboveThreshold) > 0) {
+          if (length(which(abs(data[c((1 + cliphoc1):cliphoc2)[aboveThreshold],jj]) > clipthres * 1.5)) > 0) {
+            CW[h, jj] = window2 # If there is a a value that is more than 150% the dynamic range then ignore entire block.
+          }
         }
         # Non-wear
         #hoc1 & hoc2 = edges of windows
         #window is bigger& window2 is smaller one
-        sdwacc = sd(as.numeric(data[(1 + hoc1):hoc2,jj]), na.rm = TRUE)
-        maxwacc = max(as.numeric(data[(1 + hoc1):hoc2,jj]), na.rm = TRUE)
-        minwacc = min(as.numeric(data[(1 + hoc1):hoc2,jj]), na.rm = TRUE)
+        if (nonwear_approach == "2013") {
+          indices = (1 + hoc1):hoc2
+        } else if (nonwear_approach == "2023") {
+          indices = seq((1 + hoc1), hoc2, by = ceiling(sf / 5))
+        }
+        maxwacc = max(data[indices, jj], na.rm = TRUE)
+        minwacc = min(data[indices, jj], na.rm = TRUE)
         absrange = abs(maxwacc - minwacc)
-        if (is.numeric(absrange) == TRUE & is.numeric(sdwacc) == TRUE & is.na(sdwacc) == FALSE) {
+        if (absrange < racriter) {
+          sdwacc = sd(data[indices,jj], na.rm = TRUE)
           if (sdwacc < sdcriter) {
-            if (absrange < racriter) {
-              NW[NWflag,jj] = 1
-            }
+            NW[NWflag,jj] = 1
           }
-        } else {
-          NW[NWflag,jj] = 1 # if sdwacc, maxwacc, or minwacc could not be derived then label as non-wear
         }
       }
-      CW = CW / (window2) #changed 30-1-2012, was window*sf
+      CW = CW / (window2)
       if (length(params_rawdata[["rmc.col.wear"]]) == 0) {
         NWav[h] = (NW[h,1] + NW[h,2] + NW[h,3]) #indicator of non-wear
       }
-      CWav[h] = max(c(CW[h,1],CW[h,2],CW[h,3])) #indicator of clipping
+      CWav[h] = max(c(CW[h, 1], CW[h, 2], CW[h, 3])) #indicator of clipping
     }
   } 
-
   # In NWav: single 1's surrounded by 2's or 3's --> 2 (so it is considered nonwear)
   ones = which(NWav == 1)
   if (length(ones) > 0) {

R/g.calibrate.R

@@ -324,7 +324,13 @@ g.calibrate = function(datafile, params_rawdata = c(),
       nomovement = which(meta_temp[,5] < sdcriter & meta_temp[,6] < sdcriter & meta_temp[,7] < sdcriter &
                            abs(as.numeric(meta_temp[,2])) < 2 & abs(as.numeric(meta_temp[,3])) < 2 &
                            abs(as.numeric(meta_temp[,4])) < 2) #the latter three are to reduce chance of including clipping periods
-      meta_temp = meta_temp[nomovement,]
+      if (length(nomovement) < 10) {
+        # take only one row to trigger that autocalibration is skipped 
+        # with the QCmessage that there is not enough data
+        meta_temp = meta_temp[1, ] 
+      } else {
+        meta_temp = meta_temp[nomovement,]
+      }
       dup = which(rowSums(meta_temp[1:(nrow(meta_temp) - 1), 2:7] == meta_temp[2:nrow(meta_temp), 2:7]) == 3) # remove duplicated values
       if (length(dup) > 0) meta_temp = meta_temp[-dup,]
       rm(nomovement, dup)

R/g.part5.R

@@ -276,8 +276,11 @@ g.part5 = function(datadir = c(), metadatadir = c(), f0=c(), f1=c(),
             # Add first waking up time, if it is missing:
             ts = g.part5.addfirstwake(ts, summarysleep = summarysleep_tmp2, nightsi, sleeplog, ID,
                                       Nepochsinhour, SPTE_end)
+            # Convert time column from iso8601 to POSIX regardless of whether it is aggregated
+            # to ensure the format is consistent
+            ts$time = iso8601chartime2POSIX(ts$time,tz = params_general[["desiredtz"]])
             if (params_general[["part5_agg2_60seconds"]] == TRUE) { # Optionally aggregate to 1 minute epoch:
-              ts$time_num = floor(as.numeric(iso8601chartime2POSIX(ts$time,tz = params_general[["desiredtz"]])) / 60) * 60
+              ts$time_num = floor(as.numeric(ts$time) / 60) * 60
 
               # only include angle if angle is present
               angleColName = ifelse("angle" %in% names(ts), yes = "angle", no = NULL)
@@ -410,6 +413,7 @@ g.part5 = function(datadir = c(), metadatadir = c(), f0=c(), f1=c(),
                   #-------------------------------
                   # ignore all nights in 'inights' before the first waking up and after the last waking up
                   FM = which(diff(ts$diur) == -1)
+                  SO = which(diff(ts$diur) == 1)
                   # now 0.5+6+0.5 midnights and 7 days
                   for (timewindowi in params_output[["timewindow"]]) {
                     nightsi = nightsi_bu
@@ -419,6 +423,11 @@ g.part5 = function(datadir = c(), metadatadir = c(), f0=c(), f1=c(),
                         # ignore first and last midnight because we did not do sleep detection on it
                         nightsi = nightsi[nightsi > FM[1] & nightsi < FM[length(FM)]]
                       }
+                    } else if (timewindowi == "OO") {
+                      if (length(SO) > 0) {
+                        # ignore data before the first sleep onset and after the last sleep onset
+                        nightsi = nightsi[nightsi > SO[1] & nightsi < SO[length(SO)]]
+                      }
                     } else {
                       # if first night is missing then nights needs to align with diur
                       startend_sleep = which(abs(diff(ts$diur)) == 1)
@@ -428,8 +437,10 @@ g.part5 = function(datadir = c(), metadatadir = c(), f0=c(), f1=c(),
                     }
                     if (timewindowi == "MM") {
                       Nwindows = length(nightsi) + 1
-                    } else {
+                    } else if (timewindowi == "WW") {
                       Nwindows = length(which(diff(ts$diur) == -1))
+                    } else if (timewindowi == "OO") {
+                      Nwindows = length(which(diff(ts$diur) == 1))
                     }
                     indjump = 1
                     qqq_backup = c()
@@ -442,7 +453,6 @@ g.part5 = function(datadir = c(), metadatadir = c(), f0=c(), f1=c(),
                     }
                     for (wi in 1:Nwindows) { #loop through 7 windows (+1 to include the data after last awakening)
                       # Define indices of start and end of the day window (e.g. midnight-midnight, or waking-up or wakingup
-
                       defdays = g.part5.definedays(nightsi, wi, indjump,
                                                    nightsi_bu, epochSize = ws3new, qqq_backup, ts, 
                                                    timewindowi, Nwindows, qwindow = params_247[["qwindow"]],

R/g.part5.addfirstwake.R

@@ -1,4 +1,4 @@
-g.part5.addfirstwake =function(ts, summarysleep, nightsi, sleeplog, ID, 
+g.part5.addfirstwake = function(ts, summarysleep, nightsi, sleeplog, ID, 
                                Nepochsinhour, SPTE_end) {
   # Note related to if first and last night were ignored in part 4:
   # - diur lacks the first and last night at this point in the code.
@@ -72,7 +72,7 @@ g.part5.addfirstwake =function(ts, summarysleep, nightsi, sleeplog, ID,
     }
     if (is.na(wake_night1_index)) wake_night1_index = 0
     if (wake_night1_index < firstwake & wake_night1_index > 1 & (wake_night1_index-1) > nightsi[1]) {
-      ts$diur[nightsi[1]:(wake_night1_index-1)] = 1
+      ts$diur[1:(wake_night1_index-1)] = 1
     } else {
       # Person slept only during the afternoon on day 2
       # And there is no sleep data available for the first night
@@ -82,8 +82,8 @@ g.part5.addfirstwake =function(ts, summarysleep, nightsi, sleeplog, ID,
       # and merging of the sleep variables is still consistent with
       # the other recording.
       dummywake = max(firstonset - round(Nepochsinhour/12), nightsi[1] + round(Nepochsinhour * 6))
-      ts$diur[nightsi[1]:dummywake] = 1 
-      ts$nonwear[nightsi[1]:firstonset] = 1
+      ts$diur[1:dummywake] = 1 
+      ts$nonwear[1:firstonset] = 1
     }
   }
   return(ts)