Error: shard <num>: short write

DiKa · January 12, 2021, 11:40am

Hi, I’m a newbie so sorry for possible inaccuracies in my question.

I’m using InfluxDB v1.8.3 and everything was fine before Christmas but after holidays the error in the title (“shard : short write”) appeared every time I tried to manage my db, when I tried to insert new data, to drop a measurement… I apparently fixed it switching from tsi1 to inmem (following instructions here: Upgrade to InfluxDB 1.8.x | InfluxDB OSS 1.8 Documentation) and everything works fine now but I’m not sure this really was the problem, what could have caused it?

Furthermore, searching for info about shards I’ve read here Simplifying InfluxDB: Shards and Retention Policies | InfluxData that infinite retention policy doesn’t really mean data are kept in memory endlessly but for the shard group duration (“If your retention policy is on autogen (or infinite), the data is not actually stored infinitely—this just means the retention policy matches the shard group duration, so the retention policy is effectively disabled.”) even if I can’t find a proof about that in the documentation. Is it correct?

Thank you in advance.

Anaisdg · January 19, 2021, 4:52pm

Hello @DiKa,

I don’t know what could have caused it, but it looks like it might have been an fsync issue?

github.com

influxdata/influxdb/blob/15b95312736f1ea651d2e14ba8343e483d5cd92a/tsdb/engine/tsm1/writer.go#L398


      
          			if written-lastSync > fsyncEvery {
          				if err := f.Sync(); err != nil {
          					return 0, err
          				}
          				lastSync = written
          			}
          			if ew != nil {
          				err = ew
          				break
          			}
          			if nr != nw {
          				err = io.ErrShortWrite
          				break
          			}
          		}
          		if er != nil {
          			if er != io.EOF {
          				err = er
          			}
          			break
          		}

Yes that’s true. And by your retention. policy being disabled you won’t be expiring your data so you can keep that data for as long as you want. That wording is a little confusing admittedly.

Shahar_Sadkovich · June 3, 2021, 7:54am

I just encountered this error as well when writing, what does it mean?
I can find anything about it.

Topic		Replies	Views
Indefinite Retention Store influxdb	0	721	February 4, 2019
Question about retentions Store	8	3529	February 27, 2019
Shard file created but not written Store	2	398	June 7, 2023
Write failed for shard #### field type conflict service=write Store influxdb	9	2773	August 9, 2017
Relationship between Retention Policy and Shard Group Duration InfluxDB 1 influxdb , retention-policy	4	2132	December 23, 2021

Error: shard <num>: short write

Related topics