Problem

Status Code 14: Status 14 (file write failed) occurs when updating host properties on a Windows 2008 Master Server

Error

unable to save data on some hosts. An error occured on host The error is "file write failed (14)"

Solution
Overview: Status 14 (file write failed) occurs when updating host properties on a Windows 2008 Master Server

Workaround: This has been seen in cases where User Account Control (UAC) is enabled under Control Panel User Accounts. This causes a logged in Administrator to run all programs with User-Level rights as opposed to Administrator-Level rights unless the user specifically requests to run an application with Administrator rights.

There are three workarounds: 1) If User Account Control is not required, disabling it may allow the host properties update to complete. This can present a risk to the OS and needs to be evaluated by the appropriate Windows Administrator.

2) In the short-term, to avoid this problem, you can manually launch the NetBackup Administration Console with Administrator rights by rt-clicking on the icon and selecting "Run as administrator").

3) For long-term use, you can change the short-cut icon to always launch with Administrator rights by rt-clicking on it, selecting Properties > Shortcut > Advanced and place a check in the box next to "Run as administrator".

Problem
catalog backup failed: status code 14

Solution
Overview: To facilitate disaster recovery, it is advantageous to configure the hot catalog backup policy to save a DR file at backup time for use at recovery time. If the specified location is not accessible at the time of the catalog backup, the DR file will not be saved and the backup will exit with status 14. Troubleshooting: The job details for the parent job shows this message.
begin catalog backup: database manager query operation status: 14

Resolution: 1) Inspect the hot catalog backup policy and review the pathname to which the disaster recovery file is to be written. 2) Verify that the destination is a directory, that the directory exists, and that processes can 'change directory' to the path and that the directory allows search, read, and write operations. In this instance, the directory existed but did not allow search or write permissions.

Problem
NDMP backups are failing with status 174 (media manager - system error occurred) reported. Status code 14 (file write failed) can also be seen in logs.

Solution
Issue: backup: bptm error: exits status code 174.

Error Code/Message: From the bptm log on the media server:

bptm <bptm process ID> media manager exiting because bpbrm is no longer active

The ndmpagent (VxUL oid = 134) log contains:

V-134-105 [ConnectionToBrm::WriteSocket] write socket err 32 (Broken pipe) V-134-19 [NdmpAgent::SetErrorAndHalt] ConnectionToBrm.cpp(209) - error code 14 (file write failed) [Error] V-134-112 FH send to BRM failed - Ddi - 0 0 0 0 764210 0 0 21 764208 Port_grp_name_add.htm

EVIDENCE: Entries similar to the following are in the ndmpagent log:

03/29/08 03:43:59.868 [MainStartup] ==================== STARTUP ==================== 03/29/08 03:43:59.870 [Info] V-134-2 started process ndmpagent (pid=21696) 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter stream_count 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter 4 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter stream_number 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter 1 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter -use_otm 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter -fso 03/29/08 03:43:59.870 [Warning] V-134-255 Unrecognized parameter -use_ofb 03/29/08 03:43:59.874 [NdmpAgent::CheckLogLevels] DiagLevel = 1 DebugLevel = 1 03/29/08 03:44:21.511 [NdmpGlueLogCb] ndmp_open_and_auth_via_host: hostname=myhost, via_hostname=myhost 03/29/08 03:44:21.651 [NdmpGlueLogCb] ndmp_open_and_auth_via_host: using NDMP protocol version 4 03/29/08 03:44:22.301 [NdmpFhManager::SetDarEncodeVersion] Supported NetBackup DAR version = 2 03/29/08 03:44:22.501 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: creating "/vol/vol0/../snapshot_for_backup.4296" snapshot. 03/29/08 03:44:22.841 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: Using Full Volume Dump 03/29/08 03:44:28.521 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: Date of this level 0 dump: Sat Mar 29 03:44:22 2008. 03/29/08 03:44:32.926 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: Date of last level 0 dump: the epoch. 03/29/08 03:44:32.926 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: Dumping /vol/vol0 to NDMP connection 03/29/08 03:44:32.926 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: mapping (Pass I)[regular files] 03/29/08 03:44:33.022 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: mapping (Pass II)[directories] 03/29/08 03:44:34.271 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: estimated 441525 KB. 03/29/08 03:44:35.305 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: dumping (Pass III) [directories] 03/29/08 03:44:36.901 V-134-105 [ConnectionToBrm::WriteSocket] write socket err 32 (Broken pipe)

03/29/08 03:44:38.300 V-134-19 [NdmpAgent::SetErrorAndHalt] ConnectionToBrm.cpp(209) - error code 14 (file write failed) 03/29/08 03:44:38.301 [Error] V-134-112 FH send to BRM failed - Ddi - 0 0 0 0 764210 0 0 21 764208 Port_grp_name_add.htm 03/29/08 03:44:39.298 [XmServerTape::PassMessageAndReply] attempting tape access when NAS server may be accessing it. message = 769 (NDMP_TAPE_CLOSE) 03/29/08 03:44:39.732 [Error] V-134-32 NDMP backup failed, path = /vol/vol0 03/29/08 03:44:40.421 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 DUMP: dumping (Pass IV) [regular files] 03/29/08 03:44:45.992 [NdmpGlueLogCb] myhost: NDMP_LOG_NORMAL 0 Aborted by client 03/29/08 03:44:45.992 [NdmpGlueLogCb] ndmp_wait_for_msg: notify_data_halted already arrived

From the bptm log on the media server:

03:44:36.833 [21701] <2> set_job_details: LOG 1206758676 16 bptm 21701 media manager exiting because bpbrm is no longer active 03:44:37.340 [21701] <2> check_error_history: just tpunmount: called from bptm line 17644, EXIT_Status = 174 03:44:37.340 [21701] <2> io_close: closing /usr/openv/netbackup/db/media/tpreq/drive_binding_r29_d3, from bptm.c.15486 03:44:39.298 [21701] <2> NdmpSession: ndmp_public_session_destory: destroying session 0x31a948 03:44:46.291 [21701] <2> NdmpSession: [4] Received 8 (FAILED_ALL_PATHS) "" 03:44:46.292 [21701] <2> NdmpSession: [4] Replying error = 0 03:44:46.292 [21701] <2> NdmpSession: ndmp_xm_test_echo_callback: unprocessed message 8 03:44:46.292 [21701] <2> NdmpSession: [5] Received 16 (CONNECT_CLOSE_REPLY) "" 03:44:46.292 [21701] <2> NdmpSession: [5] Replying error = 0 03:44:51.291 [21701] <2> io_close: ignore ndmp close error, NDMP_ILLEGAL_STATE_ERR

From the bpbrm log on the media server:

03:43:56.626 [21646] <2> bpbrm main: max_files_per_add = 0 ... 03:44:35.842 [21646] <2> bpbrm main: ADDED FILES TO DB FOR myhost_1206758632 1 /vol/vol0

It was discovered that the touch file MAX_FILES_PER_ADD had been created on the media server, but did not include a value:
# cat MAX_FILES_PER_ADD #

Resolution: The MAX_FILES_PER_ADD file was removed so that the default value (5000) was used. This allowed NDMP backups to succeed.
# rm /usr/openv/netbackup/MAX_FILES_PER_ADD