After nextcloud dashboard complains that I'm missing some headers for couple months, I finally look into this, and found that the way nginx handling add_header is quite annoying. It will only apply the add_header directive in deepest context match.

Ex:

server {
  location / {
    add_header xxx ooo; # I will be applied
  }
}

server {
  add_header xxx ooo; # I will be applied
  location / {
  }
}

server {
  add_header xxx ooo; # I won't be applied
  location / {
    add_header xxx ooo; # I will be applied
  }
}

and all this happens because I added cache and insert the header indicating the cache status, which makes server level add_header being ignored.