Handle search keywords containing characters outside the latin alphabet

[E5rael]

• This is not a feature request
• This is not an image-provider request
• I have searched the issuetracker if the issue is not already reported.

Describe the bug:
Images containing umlaut characters (e.g. Ä or Ö) in their tags aren't displaying.

To Reproduce:
Steps to reproduce the behavior:

1. Go to Splash settings
2. List only tags containing umlaut characters, for example jönköping, hämeenlinna, jyväskylä
3. Refresh the page, and an error background image is displayed.

Expected behavior:
Images containing these tags should appear.

Server:

• Nextcloud version: 27.0.1
• Splash-App version: 2.2.1

Additional Information:

• The bug can be circumvented by omitting the umlauts, i.e. "jonkoping, hameenlinna, jyvaskyla".
• It looks like some tags containing umlauts DO display other images besides the error image, such as öland and häme. Note, however, that the tag häme only displays an image of a cat, completely different from the 23 images that you can find, if you use the same tag at unsplash.com.
• I also tried adding cyrillic tags, such as Москва, and it seemed to work.

[timonsky]

Line 116 in lib/ProviderHandler/Provider.php is responsible for that behaviour
$term = preg_replace('/[^a-z]/i','', $term);

does not only affect umlauts, but basically any character that is not in the range U+0061 to U+007A (standard latin lowercase letters)
 

I also tried adding cyrillic tags, such as Москва, and it seemed to work.

that's because it results in an empty search string, so you just got a completely random image, not necessarily with any relation to moscow

 

I'm taking the comment line // only allow letters as searchterm as that it was a design choice with a rationale.

[joshtrichards]

Yeah, at the moment this is a deliberate design choice (probably for simplicity of implementation at the time).

So this mostly an enhancement request + probably some robustness improvements to better handle keywords containing characters outside the standard Latin alphabet.

So I'd say this is a two-parter:

• bug: handle unacceptable characters more gracefully
• enhancement: decide how to handle characters outside the Latin alphabet (taking into account possible differences across providers)