Posts

Showing posts from June, 2015

Foreman bug Warning Infinity

Yet another bug in newly installed Foreman,

While browsing diffrent parts of foreman gui, I get this error:

We're sorry, but something went wrong!

Warning! Infinity
And in the logs:
# cat /var/log/foreman/production.log Started GET "/hosts?search=last_report+%3E+%225+minutes+ago%22+and+status.enabled+%3D+true+and+status.applied+%3D+0+and+status.failed+%3D+0+and+status.pending+%3D+0" for 10.8.204.208 at 2015-06-26 14:53:12 +0200 2015-06-26 14:53:12 [I] Processing by HostsController#index as HTML 2015-06-26 14:53:12 [I]   Parameters: {"search"=>"last_report > \"5 minutes ago\" and status.enabled = true and status.applied = 0 and status.failed = 0 and status.pending = 0"} 2015-06-26 14:53:12 [I]   Rendered hosts/_list.html.erb (8.9ms) 2015-06-26 14:53:12 [I]   Rendered hosts/index.html.erb within layouts/application (12.8ms) 2015-06-26 14:53:12 [W] Operation FAILED: Infinity 2015-06-26 14:53:12 [I]   Rendered common/500.html.erb within layouts/applic…

mount: cannot remount block device /dev/mapper/vg00-lvvar read-write, is write-protected

Today one of our production systems filesystem /var went into read-only mode.

This is most probably because of underlying storage device issue, as we can see all the following errors in /var/log/messages:

Jul 2 12:08:53 hostname kernel: sd 2:0:2:0: [sdc] Unhandled error code
Jul 2 12:08:53 hostname kernel: sd 2:0:2:0: [sdc] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Jul 2 12:08:53 hostname kernel: sd 2:0:2:0: [sdc] CDB: Write(10): 2a 00 68 c8 23 17 00 00 08 00
Jul 2 12:08:54 hostname kernel: Buffer I/O error on device dm-4, logical block 2163547
Jul 2 12:08:54 hostname kernel: lost page write due to I/O error on dm-4
Jul 2 12:08:54 hostname kernel: BUG: soft lockup - CPU#2 stuck for 67s! [udisks-daemon:10901]
Jul 2 12:08:54 hostname kernel: Modules linked in: bridge nfs lockd fscache auth_rpcgss nfs_acl sunrpc fuse autofs4 arc4 ecb md4 nls_utf8 cifs target_core_iblock target_core_file target_core_pscsi target_core_mod configfs bnx2fc cnic uio fcoe libfcoe libfc scsi_transport_fc…