Skip to content

Commit

Permalink
docs: update docusaurus to v3 (#2163)
Browse files Browse the repository at this point in the history
  • Loading branch information
B4nan authored Nov 2, 2023
1 parent 430f025 commit 9aa8593
Show file tree
Hide file tree
Showing 4 changed files with 2,495 additions and 1,851 deletions.
2 changes: 1 addition & 1 deletion packages/http-crawler/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Provides a framework for the parallel crawling of web pages using plain HTTP requests. The URLs to crawl are fed either from a static list of URLs or from a dynamic queue of URLs enabling recursive crawling of websites.

It is very fast and efficient on data bandwidth. However, if the target website requires JavaScript to display the content, you might need to use {[PuppeteerCrawler](https://crawlee.dev/api/puppeteer-crawler/class/PuppeteerCrawler) or [PlaywrightCrawler](https://crawlee.dev/api/playwright-crawler/class/PlaywrightCrawler) instead, because it loads the pages using full-featured headless Chrome browser. **This crawler downloads each URL using a plain HTTP request and doesn't do any HTML parsing.**
It is very fast and efficient on data bandwidth. However, if the target website requires JavaScript to display the content, you might need to use [PuppeteerCrawler](https://crawlee.dev/api/puppeteer-crawler/class/PuppeteerCrawler) or [PlaywrightCrawler](https://crawlee.dev/api/playwright-crawler/class/PlaywrightCrawler) instead, because it loads the pages using full-featured headless Chrome browser. **This crawler downloads each URL using a plain HTTP request and doesn't do any HTML parsing.**

The source URLs are represented using [Request](https://crawlee.dev/api/core/class/Request) objects that are fed from [RequestList](https://crawlee.dev/api/core/class/RequestList) or [RequestQueue](https://crawlee.dev/api/core/class/RequestQueue) instances provided by the [HttpCrawlerOptions.requestList](https://crawlee.dev/api/http-crawler/interface/HttpCrawlerOptions#requestList) or [HttpCrawlerOptions.requestQueue](https://crawlee.dev/api/http-crawler/interface/HttpCrawlerOptions#requestQueue) constructor options, respectively.

Expand Down
10 changes: 5 additions & 5 deletions website/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -130,10 +130,10 @@ module.exports = {
id: 'GTM-5P7MCS7',
},
],
async function runnableCodeBlock(context, options) {
async function runnableCodeBlock() {
return {
name: 'runnable-code-block',
configureWebpack(config, isServer) {
configureWebpack() {
return {
resolveLoader: {
alias: {
Expand Down Expand Up @@ -225,9 +225,9 @@ module.exports = {
},
prism: {
defaultLanguage: 'typescript',
theme: require('prism-react-renderer/themes/github'),
darkTheme: require('prism-react-renderer/themes/dracula'),
additionalLanguages: ['docker', 'log'],
theme: require('prism-react-renderer').themes.github,
darkTheme: require('prism-react-renderer').themes.dracula,
additionalLanguages: ['docker', 'log', 'bash', 'diff', 'json'],
},
metadata: [],
image: 'img/crawlee-og.png',
Expand Down
22 changes: 13 additions & 9 deletions website/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
"devDependencies": {
"@apify/eslint-config-ts": "^0.4.0",
"@apify/tsconfig": "^0.1.0",
"@docusaurus/module-type-aliases": "3.0.0",
"@docusaurus/types": "3.0.0",
"@types/react": "^18.0.28",
"@typescript-eslint/eslint-plugin": "^6.0.0",
"@typescript-eslint/parser": "^6.0.0",
Expand All @@ -26,26 +28,28 @@
"path-browserify": "^1.0.1",
"prettier": "^3.0.0",
"rimraf": "^5.0.0",
"typescript": "5.1.6"
"typescript": "5.2.2"
},
"dependencies": {
"@apify/utilities": "^2.8.0",
"@docusaurus/core": "2.4.1",
"@docusaurus/plugin-client-redirects": "2.4.1",
"@docusaurus/preset-classic": "2.4.1",
"@giscus/react": "^2.2.6",
"@docusaurus/core": "3.0.0",
"@docusaurus/plugin-client-redirects": "3.0.0",
"@docusaurus/preset-classic": "3.0.0",
"@giscus/react": "^2.3.0",
"@mdx-js/react": "^3.0.0",
"axios": "^1.5.0",
"buffer": "^6.0.3",
"clsx": "^2.0.0",
"crypto-browserify": "^3.12.0",
"docusaurus-gtm-plugin": "^0.0.2",
"docusaurus-plugin-typedoc-api": "3.0.0",
"docusaurus-plugin-typedoc-api": "4.0.0",
"prism-react-renderer": "^2.1.0",
"process": "^0.11.10",
"prop-types": "^15.8.1",
"raw-loader": "^4.0.2",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"react-lite-youtube-embed": "^2.3.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-lite-youtube-embed": "^2.3.52",
"stream-browserify": "^3.0.0",
"unist-util-visit": "^5.0.0"
},
Expand Down
Loading

0 comments on commit 9aa8593

Please sign in to comment.