Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Json data returned from imdb? #200

Open
copperhead57 opened this issue Dec 14, 2023 · 4 comments
Open

Use Json data returned from imdb? #200

copperhead57 opened this issue Dec 14, 2023 · 4 comments
Assignees
Labels
enhancement Enhancements and improvements, nice to haves for a next release but never mandatory. feedback-wanted Looking for feedback on the issue at hand. Opinions please, constructive ones preferred.
Milestone

Comments

@copperhead57
Copy link
Collaborator

imdb have introduced the use of json data in the returned page holding most of the data that is currently extracted by interrupting the html code.

I have a version in my branch json-blob-imdb using this json data where possible.

any feedback and testing is welcomed.

@copperhead57 copperhead57 added enhancement Enhancements and improvements, nice to haves for a next release but never mandatory. feedback-wanted Looking for feedback on the issue at hand. Opinions please, constructive ones preferred. labels Dec 14, 2023
@copperhead57 copperhead57 self-assigned this Dec 14, 2023
@johanneskonst johanneskonst added this to the 4.2 milestone Jan 29, 2024
@johanneskonst
Copy link
Collaborator

Your changes look great but collide with my planning ;-)

I had initially opened #152 for this, found #177 to be more pressing as to make a stable foundation in figuring out why something breaks, had a look at #19, #72 and #106 to come to the conclusion that the engines part needed quite some love, and then life happened ...

Would it be wise to make a small plan on the engines we want to support, perhaps give it a base class perhaps for sane defaults, and then build/rebuild from there? I lost my work from beginning '23 but might be able to recreate something...

@benausro
Copy link

benausro commented Apr 5, 2024

Imdb parsing works as long as they don't change anything on the site, which they constantly do. I fill my videodb installation with data from tmdb as they provide an api which makes it really easy to gather information. Wouldn't that be the easier and more sustainable way?
(I am using python to scrape the data for videodb, I am no php developer, sorry)

@robelix
Copy link
Contributor

robelix commented Jan 6, 2025

I worked a bit on copperhead57s json-blob-imdb2

https://github.com/robelix/videodb/tree/json-blob-imdb2

rebased it to master
fixed a lot of tests
small fix in the imdb engine to avoid an exception

Looks quite good to me - all the data gets imported.
5 tests are still failing - I'll have a look to these another time.

@copperhead57
Copy link
Collaborator Author

@robelix,
well done in refining code.
if you and others are happy, can you create a pull request.
he last 5 test can be done when ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancements and improvements, nice to haves for a next release but never mandatory. feedback-wanted Looking for feedback on the issue at hand. Opinions please, constructive ones preferred.
Projects
None yet
Development

No branches or pull requests

4 participants