Logstash has encountered a problem

[ghost]

Hi @cookandy can you explain why this problem occur?

[2020-01-03T22:05:36,143][ERROR][logstash.javapipeline ] A plugin had an unrecoverable error. Will restart this plugin.
logstash_1 | Pipeline_id:main
logstash_1 | Plugin: <LogStash::Inputs::File mode=>"read", path=>["/logstash-logs///*.gz"], id=>"f84d42e863f80367cc2041a270892e6cecccc2b922eb21fd67af169fced8c5d5", type=>"gzip", file_completed_action=>"delete", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_b39987bc-e148-4a65-bb87-1fad8c86dd5d", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, start_position=>"end", delimiter=>"\n", close_older=>3600.0, sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc">
logstash_1 | Error: Unexpected end of ZLIB input stream
logstash_1 | Exception: Java::JavaIo::EOFException
logstash_1 | Stack: java.util.zip.InflaterInputStream.fill(java/util/zip/InflaterInputStream.java:245)
logstash_1 | java.util.zip.InflaterInputStream.read(java/util/zip/InflaterInputStream.java:159)
logstash_1 | java.util.zip.GZIPInputStream.read(java/util/zip/GZIPInputStream.java:118)
logstash_1 | sun.nio.cs.StreamDecoder.readBytes(sun/nio/cs/StreamDecoder.java:284)
logstash_1 | sun.nio.cs.StreamDecoder.implRead(sun/nio/cs/StreamDecoder.java:326)
logstash_1 | sun.nio.cs.StreamDecoder.read(sun/nio/cs/StreamDecoder.java:178)
logstash_1 | java.io.InputStreamReader.read(java/io/InputStreamReader.java:185)
logstash_1 | java.io.BufferedReader.fill(java/io/BufferedReader.java:161)
logstash_1 | java.io.BufferedReader.readLine(java/io/BufferedReader.java:326)
logstash_1 | jdk.internal.reflect.GeneratedMethodAccessor25.invoke(jdk/internal/reflect/GeneratedMethodAccessor25)
logstash_1 | jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
logstash_1 | java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)
logstash_1 | org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:455)
logstash_1 | org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:316)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.read_mode.handlers.read_zip_file.handle_specifically(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/read_mode/handlers/read_zip_file.rb:26)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.read_mode.handlers.base.handle(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/read_mode/handlers/base.rb:26)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.read_mode.processor.read_zip_file(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/read_mode/processor.rb:39)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.read_mode.processor.process_active(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/read_mode/processor.rb:102)
logstash_1 | org.jruby.RubyArray.each(org/jruby/RubyArray.java:1800)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.read_mode.processor.process_active(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/read_mode/processor.rb:88)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.read_mode.processor.process_all_states(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/read_mode/processor.rb:45)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.watch.iterate_on_state(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/watch.rb:67)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.watch.subscribe(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/watch.rb:45)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.filewatch.observing_read.subscribe(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/filewatch/observing_read.rb:12)
logstash_1 | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_file_minus_4_dot_1_dot_11.lib.logstash.inputs.file.run(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.11/lib/logstash/inputs/file.rb:339)
logstash_1 | usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.inputworker(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:314)
logstash_1 | usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0$VARARGS(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
logstash_1 | usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_input(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:306)
logstash_1 | org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)
logstash_1 | org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)
logstash_1 | org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)
logstash_1 | java.lang.Thread.run(java/lang/Thread.java:834)

I need help.. I'm no expert in logstash in particular, so please excuse if this question was already posted (solved?) somewhere else.

[cookandy]

@slovenski it seems like the archive hasn't finished downloading before it tries to ingest into logstash

logstash_1 | Error: Unexpected end of ZLIB input stream
logstash_1 | Exception: Java::JavaIo::EOFException

If you want to test, you can try to open the archive manually and see if it works. You might have to adjust the frequency of your log pulls.