About Series

[enigmalea]

I'm about to start work on series scraping and just wanted to see if there was anything else to add to our list from issue #36. Reformatted to make it easier to read.

• series id
• series url
• series name
• series creators
  • username
  • pseud
  • url (?)
• series begin date
• series update date
• series description
• series notes
• series stats
  • series word count
  • series works counts
  • series completion status
  • series bookmarks count
• series works
  • series works titles
  • series works urls

Questions/Discussion/Thoughts Needed:
Because the series page has works summaries it is theoretically possible to scrap things like stats, summaries, creators, tags, etc, from the individual works that make up the series. Can anyone think of any reason or use case to do this? It's not on my list because I didn't use these things on my first iteration for Archivist, but maybe it's worth scraping for another use that I haven't considered.

[essential-randomness]

Questions:

1. Is the work info you can scrape from series the same as the one we have in the individual work
2. If it's not the same, what are we missing? Are there other pages where this is the only information displayed?

[essential-randomness]

(pressed enter too soon)

For Series, I would say workList (which I would just call works) should be of type WorkSummary[], no?

If there is indeed information that we cannot get in that view, we should consider whether there's other views where that information cannot be gotten, and whether it should be abstracted in another generic type or special-cased for the summary page.

[enigmalea]

For Series, I would say workList (which I would just call works) should be of type WorkSummary[], no?

No.

Works in a series have a full card, just like when you navigate through searches or a fandom tag or whatever.

Ideally a works list would be a list of the titles and/or urls for the works that make up the series.

but since these full cards are there you can ALSO scrape some info BEYOND just the title and URL of the work.

Unless I'm misunderstanding what WorkSummary is and it's more than just the thing typed in the Summary box.

[essential-randomness]

This is WorkSummary: https://github.com/essential-randomness/AO3.js/blob/dca271ccd033091f9573f48c4b7e147acf46fc5f/src/types/entities.ts#L65

There's more than the title and URL of the work, and I think it has all the info that are there in the card.

[enigmalea]

Hmmm... yes. but.... the series page has... many of those - one for each work.

[riazaia]

[...] I think for WorkSummary the series information would have to be WorkSeries instead of WorkSeries[].

I made it an array because works can belong to multiple series. So, using this work link as an example, and using the poorly-named WorkSeries type I proposed - this work returns:

[
  {
    "id": "3096093",
    "name": "Mirror Jadzia Dax/Mirror Lenara Kahn One-Shots",
    "part": "2",
    "prevId": "41470299",
    "nextId": null
  },
  {
    "id": "3101340",
    "name": "Whumptember Prompts 2022",
    "part": "6",
    "prevId": "41544159",
    "nextId": null
  }
]

You get this information from visiting the individual work page, no need to click through to the series page. I believe I should be less extra and type it as:

export interface WorkSeries {
        id:  string;
        title: string;
        part: number;
}

and coincidentially that's the information you can also get from the blurb.

---

For Series, I would say workList (which I would just call works) should be of type WorkSummary[], no?

I'd need to look carefully at the type and see what it includes or doesn't. Because the type WorkSummary is currently being used to return the information you get from the individual work page, and I didn't realise we would use it for the blurbs. Most things appear in both places, at a glance, with the way the interface is currently written, the only thing we can't get without going to the individual work page is the publishedAt date. That being said, though:

• there are indeed things you get from the work page that you can't from the blurbs (though that might be more appropiate in a different discussion?)
• to get the information from the blurbs we'd naturally need to write new methods

As a note, I keep saying blurb but I dunno if that's clear that this is what I'm referring to:

[enigmalea]

As a note, I keep saying blurb but I dunno if that's clear that this is what I'm referring to:

I called it a card, but yes.

I forgot that PublishedAt wasn't there. I think it's... the only missing thing from the WorkSummary type, and the fact there's multiple of them on a single series page, so it would need to be an Array or Nested Object, if we returned all the info from each work card/blurb/thing.

It's also worth nothing that a series with a significant number of works will paginate.

[enigmalea]

So currently, Archivist can return a works list (just links to the works in the series) or at least as much of one as fits in a discord embed, so I think that's all I'm worried about, so title and url are my only use case.

BUT... I'm not sure what use cases someone else might have.
Maybe... using a tool to bookmark all the works in a series in raindrop.io and wanting to just be able to drop the series into an input and have all the tags and summary and such added?

IDK.

[essential-randomness]

Blurb might be a better term than Summary TBH.

I think what we should have is

interface WorkBlurb {
  // What is now WorkSummary, but without the info missing
}

interface Work extends WorkBlurb {
  // All the data in the work page that is not already in WorkBlurb
  // NOTE: the extends keyword means Work has all of WorkBlurb's fields
}

Alternatively our type for WorkBlurb in the series part can be Omit<WorkBlurb, "publishedAt"> (that is, everything in WorkBlurb, except "publishedAt").

[enigmalea]

No thoughts other than "TypeScript is too powerful."

[essential-randomness]

For the "how do we fetch a series if it's too long problem we can do this:

interface Series {
  // ...
  works: {
    currentPage: WorkBlurb[],
    nextPage: string | null;
    previousPage: string | null;
  }
}

next page/previous would be the the next page of work or previous page of work, if they exist. This is a common pattern for stuff with pagination.

[essential-randomness]

No thoughts other than "TypeScript is too powerful."

This isn't even its final form