Commit 08fd4f85 authored by Alex Tayts's avatar Alex Tayts
Browse files

add file/directory check

parent 7f19a909

CHANGELOG.md

+4 −0
Original line number Diff line number Diff line
@@ -2,6 +2,10 @@ 


All notable changes to this project will be documented in this file.



## Release 0.4.0



* Added file/directory check.



## Release 0.3.1



* Fixed a bug with long process names.

README.md

+38 −2
Original line number Diff line number Diff line
@@ -78,18 +78,54 @@ server_patching::validate::exports: 
server_patching::validate::zfs_pools:

  - pool1

  - pool2

server_patching::validate::files:

  - path: /var/log/application.log

    exists: true

    type: regular file # file types are as outputted by stat(1)

    owner: application

    group: operations

    mode: 644

    age: 5m # maximum modification time format XXdXXhXXmXXs with any omited

    min_size: 10

    max_size: 6M

    contains:

      - application startup success

      - connection established

    not_contains:

      - startup failed

      - exception 24B70 in object.getConnected

    last_lines: 100 # or 'first_lines: 100', but not both

  - path: /srv/data/project

    type: directory

    owner: pete

    group: staff

    min_size: 5M # allocated space, calculation may take time

    max_size: 10G # allocated space, calculation may take time

```



The fragment generates verifies if:

* VMWare tools service is running

* CrowdStrike daemon process is running

* Java process which has *solr* in its command line is running

* NetDB URL redirects somewhere (to WebAuth presumably)

* NetDB server 171.67.5.154 status URL returns success

* NetDB status URL returns success

* SSH port is open over IPv4

* Telnet port is closed over IPv4

* Given ZFS pools are imported

* Given NFS shares are exported

* Verfies that application log file

    * exists

    * is a regular file ("directory" type is supported as well)

    * has correct permissions

    * has been modified within last 5 min

    * larger than 10 bytes, but smaller than 6MB

    * contains both strings "application startup success" and "connection established" (Perl regular expressions supported)

    * contains neither "startup failed" nor "exception 24B70 in object.getConnectd"

    * only last 100 lines in the file are searched for these strings

* Checks if a project directory:

    * exists

    * has correct premissions

    * has an allocated size between 5MB and 10GB

    * (text searches within directory files not supported)



The module would generate a validation script `validation.sh` in

`/usr/local/bin/` directory, which can be triggered by SSM and return a zero

REFERENCE.md

+154 −39
Original line number Diff line number Diff line
@@ -7,7 +7,7 @@ 
### Classes



* [`server_patching`](#server_patching): A class to deploy patching-related tools

* [`server_patching::validate`](#server_patchingvalidate): Validate the server state based on predefined checks.

* [`server_patching::validate`](#server_patching--validate): Validate the server state based on predefined checks.



## Classes
@@ -23,7 +23,7 @@ At the moment only includes the class `server_patching::validate`. 
include server_patching

```



### <a name="server_patchingvalidate"></a>`server_patching::validate`

### <a name="server_patching--validate"></a>`server_patching::validate`



The class generates or removes the validation script for the

state of the server. The resulting script can be automatically
@@ -64,6 +64,29 @@ server_patching::validate::exports: 
server_patching::validate::zfs_pools:

  - pool1

  - pool2

server_patching::validate::files:

  - path: /var/log/application.log

    exists: true

    type: regular file # file types are as outputted by stat(1)

    owner: application

    group: operations

    mode: 644

    age: 5m # maximum modification time format XXdXXhXXmXXs with any omited

    min_size: 10

    max_size: 6M

    contains:

      - application startup success

      - connection established

    not_contais:

      - application failed

      - error 2740

    last_lines: 100 # or 'first_lines: 100', but not both

  - path: /srv/data/project

    type: directory

    owner: pete

    group: staff

    min_size: 5M # allocated space, calculation may take time

    max_size: 10G # allocated space, calculation may take time

```



#### Examples
@@ -78,18 +101,19 @@ include server_patching::validate 


The following parameters are available in the `server_patching::validate` class:



* [`ensure`](#ensure)

* [`validation_script`](#validation_script)

* [`use_remctl`](#use_remctl)

* [`services`](#services)

* [`processes`](#processes)

* [`urls`](#urls)

* [`ports`](#ports)

* [`mounts`](#mounts)

* [`exports`](#exports)

* [`zfs_pools`](#zfs_pools)

* [`ensure`](#-server_patching--validate--ensure)

* [`validation_script`](#-server_patching--validate--validation_script)

* [`wait`](#-server_patching--validate--wait)

* [`services`](#-server_patching--validate--services)

* [`processes`](#-server_patching--validate--processes)

* [`urls`](#-server_patching--validate--urls)

* [`ports`](#-server_patching--validate--ports)

* [`mounts`](#-server_patching--validate--mounts)

* [`exports`](#-server_patching--validate--exports)

* [`zfs_pools`](#-server_patching--validate--zfs_pools)

* [`files`](#-server_patching--validate--files)



##### <a name="ensure"></a>`ensure`

##### <a name="-server_patching--validate--ensure"></a>`ensure`



Data type: `Enum['absent','present']`
@@ -97,7 +121,7 @@ Deploy or remove the validation script. 


Default value: `'present'`



##### <a name="validation_script"></a>`validation_script`

##### <a name="-server_patching--validate--validation_script"></a>`validation_script`



Data type: `Stdlib::Unixpath`
@@ -105,15 +129,23 @@ Location of the validation script on the system. 


Default value: `'/usr/local/bin/validate.sh'`



##### <a name="use_remctl"></a>`use_remctl`

##### <a name="-server_patching--validate--wait"></a>`wait`



Whether to create a remctl for validation script or not.

Data type: `Optional[Integer[0]]`



##### <a name="services"></a>`services`

Wait for specified number of seconds before starting checks.



Data type: `Array[Struct[{

Default value: `undef`



##### <a name="-server_patching--validate--services"></a>`services`



Data type:



```puppet

Array[Struct[{

    'name'         => String,

    'active'     => Optional[Boolean]}]]`

    'active'       => Optional[Boolean]}]]

```



Array of hashes, where each hash is describing a service and

its desired state.
@@ -127,12 +159,16 @@ Defaults to `true`. 


Default value: `[]`



##### <a name="processes"></a>`processes`

##### <a name="-server_patching--validate--processes"></a>`processes`



Data type:



Data type: `Array[Struct[{

```puppet

Array[Struct[{

    'name'         => String,

    'command'      => Optional[String],

    'running'    => Optional[Boolean]}]]`

    'running'      => Optional[Boolean]}]]

```



Array of hashes, where each hash describes the name of a process

and whether it is supposed to be running.
@@ -146,12 +182,16 @@ or `false` if it is not supposed to. Defaults to `true`. 


Default value: `[]`



##### <a name="urls"></a>`urls`

##### <a name="-server_patching--validate--urls"></a>`urls`



Data type:



Data type: `Array[Struct[{

```puppet

Array[Struct[{

    'url'          => Stdlib::HTTPUrl,

    'resolve_to'   => Optional[Stdlib::IP::Address],

    'status'     => Optional[Integer[100,510]]}]]`

    'status'       => Optional[Integer[100,510]]}]]

```



Array of hashes, where each hash describes the web URL

and the HTTP status code it is supposed to return
@@ -164,13 +204,17 @@ Options: 


Default value: `[]`



##### <a name="ports"></a>`ports`

##### <a name="-server_patching--validate--ports"></a>`ports`



Data type:



Data type: `Array[Struct[{

```puppet

Array[Struct[{

    'port'         => Stdlib::Port,

    'proto'        => Optional[Enum['tcp','udp']],

    'ip_ver'       => Optional[Enum['ipv4','ipv6']],

    'listening'  => Optional[Boolean]}]]`

    'listening'    => Optional[Boolean]}]]

```



Array of hashes, where each hash contains a port number, protocol,

IP protocol version (IPv4/IPv6) and whether anything is expected
@@ -186,7 +230,7 @@ the port is not supposed to respond. Defaults to `true`. 


Default value: `[]`



##### <a name="mounts"></a>`mounts`

##### <a name="-server_patching--validate--mounts"></a>`mounts`



Data type: `Array[Stdlib::Unixpath]`
@@ -194,7 +238,7 @@ Array of mounts to be checked for presence. 


Default value: `[]`



##### <a name="exports"></a>`exports`

##### <a name="-server_patching--validate--exports"></a>`exports`



Data type: `Array[Stdlib::Unixpath]`
@@ -202,7 +246,7 @@ Array of NFS exports to be checked for presence. Only local exports are checked. 


Default value: `[]`



##### <a name="zfs_pools"></a>`zfs_pools`

##### <a name="-server_patching--validate--zfs_pools"></a>`zfs_pools`



Data type: `Array[String]`
@@ -210,3 +254,74 @@ Array of ZFS pools to be checked for presence. 


Default value: `[]`



##### <a name="-server_patching--validate--files"></a>`files`



Data type:



```puppet

Array[Struct[{

    'path'         => Stdlib::Unixpath,

    'type'         => Optional[String[1]],

    'exists'       => Optional[Boolean],

    'owner'        => Optional[String[1]],

    'group'        => Optional[String[1]],

    'mode'         => Optional[Variant[String[3,4], Integer[0,3777]]],

    'age'          => Optional[Pattern[/^(\d{0,}d)?(\d{0,2}h)?(\d{0,2}m)?(\d{0,2}s)?$/]],

    'min_size'     => Optional[Variant[String, Integer[1]]],

    'max_size'     => Optional[Variant[String, Integer[1]]],

    'contains'     => Optional[Array[String[1]]],

    'not_contains' => Optional[Array[String[1]]],

    'first_lines'  => Optional[Integer[1]],

    'last_lines'   => Optional[Integer[1]]}]]

```



Array of hashes specifying requirements for a file or directory



Options:



* **:path** `Stdlib::Unixpath`: Full file system path to a file or directory. Mandatory.

* **Optional** `String`: :type

Type of a file or directory as shown by stat(1). The most typical ones are

`regular file`, 'directory', 'block special file', 'character specila file',

'fifo', 'socket', 'regular empty file', etc.

* **Optional** `Boolean`: :exists

Tells if the file should exist or not.

* **Optional** `String`: :owner

Owner of a file or directory. UIDs not accepted, only resolved user names.

* **Optional** `String`: :group

Group of a file or directory. GIDs not accepted, only resolved group names.

* **Optional** `String`: :mode

Mode of a file or directory either as an integer or a string.

* **Optional** `Pattern`: :age

File or directory modification date should be more recent than a given age.

Age is specified in a format XXXXdXXhXXmXXs, where X are digits and `d` stands

for days, `h` for hours, `m` for minutes and `s` for seconds. Any of these

can be omited, like `5m` or `1h15m`.

* **Optional** `Variant[String,Integer]`: :min_size

Size of the file should exceed this value. Size is specified by default in bytes.

'K', 'M', 'G', 'T' Multipliers can be appended to signify decimal Kilobytes,

Megabytes, Gigabytes and Terabytes.

* **Optional** `Variant[String,Integer]`: :max_size

Size of the file should not exceed this value. Size is specified by default in bytes.

'K', 'M', 'G', 'T' Multipliers can be appended to signify decimal Kilobytes,

Megabytes, Gigabytes and Terabytes.

* **Optional** `Array[String]`: :contains

An array of strings, all of which should be present in a text file. Perl regular

expressions are supported as well. Check fails if any of the strings is not found.

* **Optional** `Array[String]`: :not_contains

An array of strings, none of which should be present in a text file. Perl regular

expressions are supported as well. Check fails if any of the strings is found.

* **Optional** `Integer`: :first_lines

Only this number of lines counting from the beginning of a text file are

searched for strings listed in `contains` parameter. Even if a string is present

in a file, but is located later than this line number, the check would fail.

This paremeter cannot be specified together with `last_lines`.

* **Optional** `Integer`: :last_lines

Only this number of lines counting from the end of a text file are

searched for strings listed in `contains` parameter. Even if a string is present

in a file, but is located later than this line number, the check would fail.

This paremeter cannot be specified together with `first_lines`. If you do that,

this parameter would be ignored in favor of `first_lines`.



Default value: `[]`

manifests/validate.pp

+126 −16
Original line number Diff line number Diff line
@@ -40,6 +40,29 @@ 
# server_patching::validate::zfs_pools:

#   - pool1

#   - pool2

# server_patching::validate::files:

#   - path: /var/log/application.log

#     exists: true

#     type: regular file # file types are as outputted by stat(1)

#     owner: application

#     group: operations

#     mode: 644

#     age: 5m # maximum modification time format XXdXXhXXmXXs with any omited

#     min_size: 10

#     max_size: 6M

#     contains:

#       - application startup success

#       - connection established

#     not_contais:

#       - application failed

#       - error 2740

#     last_lines: 100 # or 'first_lines: 100', but not both

#   - path: /srv/data/project

#     type: directory

#     owner: pete

#     group: staff

#     min_size: 5M # allocated space, calculation may take time

#     max_size: 10G # allocated space, calculation may take time

# ```

#

# @param ensure
@@ -48,8 +71,8 @@ 
# @param validation_script

#   Location of the validation script on the system.

#

# @param use_remctl

#   Whether to create a remctl for validation script or not.

# @param wait

#   Wait for specified number of seconds before starting checks.

#

# @param services

#   Array of hashes, where each hash is describing a service and
@@ -117,12 +140,73 @@ 
# @param zfs_pools

#   Array of ZFS pools to be checked for presence.

#

# @param files

#   Array of hashes specifying requirements for a file or directory

#

# @option files [Stdlib::Unixpath] :path

#   Full file system path to a file or directory. Mandatory.

#

# @option files Optional[String] :type

#   Type of a file or directory as shown by stat(1). The most typical ones are

#   `regular file`, 'directory', 'block special file', 'character specila file',

#   'fifo', 'socket', 'regular empty file', etc.

#

# @option files Optional[Boolean] :exists

#   Tells if the file should exist or not.

#

# @option files Optional[String] :owner

#   Owner of a file or directory. UIDs not accepted, only resolved user names.

#

# @option files Optional[String] :group

#   Group of a file or directory. GIDs not accepted, only resolved group names.

#

# @option files Optional[String] :mode

#   Mode of a file or directory either as an integer or a string.

#

# @option files Optional[Pattern] :age

#   File or directory modification date should be more recent than a given age.

#   Age is specified in a format XXXXdXXhXXmXXs, where X are digits and `d` stands

#   for days, `h` for hours, `m` for minutes and `s` for seconds. Any of these

#   can be omited, like `5m` or `1h15m`.

#

# @option files Optional[Variant[String,Integer]] :min_size

#   Size of the file should exceed this value. Size is specified by default in bytes.

#   'K', 'M', 'G', 'T' Multipliers can be appended to signify decimal Kilobytes,

#   Megabytes, Gigabytes and Terabytes.

#

# @option files Optional[Variant[String,Integer]] :max_size

#   Size of the file should not exceed this value. Size is specified by default in bytes.

#   'K', 'M', 'G', 'T' Multipliers can be appended to signify decimal Kilobytes,

#   Megabytes, Gigabytes and Terabytes.

#

# @option files Optional[Array[String]] :contains

#   An array of strings, all of which should be present in a text file. Perl regular

#   expressions are supported as well. Check fails if any of the strings is not found.

#

# @option files Optional[Array[String]] :not_contains

#   An array of strings, none of which should be present in a text file. Perl regular

#   expressions are supported as well. Check fails if any of the strings is found.

#

# @option files Optional[Integer] :first_lines

#   Only this number of lines counting from the beginning of a text file are

#   searched for strings listed in `contains` parameter. Even if a string is present

#   in a file, but is located later than this line number, the check would fail.

#   This paremeter cannot be specified together with `last_lines`.

#

# @option files Optional[Integer] :last_lines

#   Only this number of lines counting from the end of a text file are

#   searched for strings listed in `contains` parameter. Even if a string is present

#   in a file, but is located later than this line number, the check would fail.

#   This paremeter cannot be specified together with `first_lines`. If you do that,

#   this parameter would be ignored in favor of `first_lines`.

#

# @example

#   include server_patching::validate

#

class server_patching::validate (

  Enum['absent','present'] $ensure = 'present',

  Stdlib::Unixpath $validation_script = '/usr/local/bin/validate.sh',

  Optional[Integer[0]] $wait = undef,

  Array[Struct[{

    'name'         => String,

    'active'       => Optional[Boolean]}]] $services = [],
@@ -142,9 +226,23 @@ class server_patching::validate ( 
  Array[Stdlib::Unixpath] $mounts = [],

  Array[Stdlib::Unixpath] $exports = [],

  Array[String] $zfs_pools = [],

  Array[Struct[{

    'path'         => Stdlib::Unixpath,

    'type'         => Optional[String[1]],

    'exists'       => Optional[Boolean],

    'owner'        => Optional[String[1]],

    'group'        => Optional[String[1]],

    'mode'         => Optional[Variant[String[3,4], Integer[0,3777]]],

    'age'          => Optional[Pattern[/^(\d{0,}d)?(\d{0,2}h)?(\d{0,2}m)?(\d{0,2}s)?$/]],

    'min_size'     => Optional[Variant[String, Integer[1]]],

    'max_size'     => Optional[Variant[String, Integer[1]]],

    'contains'     => Optional[Array[String[1]]],

    'not_contains' => Optional[Array[String[1]]],

    'first_lines'  => Optional[Integer[1]],

    'last_lines'   => Optional[Integer[1]]}]] $files = [],

) {



  if [$services, $processes, $urls, $ports, $mounts, $exports, $zfs_pools].any |$param| { !empty($param) } {

  if [$services, $processes, $urls, $ports, $mounts, $exports, $zfs_pools, $files].any |$param| { !empty($param) } {

    concat { $validation_script:

      ensure => $ensure,

      mode   => '0700',
@@ -153,7 +251,7 @@ class server_patching::validate ( 
    concat::fragment { 'bash-header':

      target  => $validation_script,

      order   => '01',

      content => epp('server_patching/validate/header.epp'),

      content => epp('server_patching/validate/header.epp', { 'wait' => $wait }),

    }



    concat::fragment { 'bash-footer':
@@ -237,6 +335,18 @@ class server_patching::validate ( 
        content => epp('server_patching/validate/zpool.epp', { 'zfs_pools' => $zfs_pools }),

      }

    }



    if !empty($files) {

      $_files = $files.map |$file| {

        { 'exists' => true } + $file

      }



      concat::fragment { 'validate-files':

        target  => $validation_script,

        order   => '08',

        content => epp('server_patching/validate/files.epp', { 'files' => $_files }),

      }

    }

  } else {

    # If no checks are defined, no reason to keep an empty

    # validation script around.

metadata.json

+3 −3
Original line number Diff line number Diff line 
{

  "name": "ruthenium-server_patching",

  "version": "0.3.1",

  "version": "0.3.0",

  "author": "Ruthenium",

  "summary": "Generate bash script to valideate server state.",

  "license": "MIT",
@@ -50,9 +50,9 @@ 
    {

      "operatingsystem": "Ubuntu",

      "operatingsystemrelease": [

        "18.04",

        "20.04",

        "22.04"

        "22.04",

        "24.04"

      ]

    }

  ],