GCP internal provider functions¶

libcloudforensics.providers.gcp.internal.build module¶

Google Cloud Build functionalities.

class libcloudforensics.providers.gcp.internal.build.GoogleCloudBuild(project_id)¶

Bases: object

Class to call Google Cloud Build APIs.

Dictionary objects content can be found in https://cloud.google.com/cloud-build/docs/api/reference/rest/v1/projects.builds

BlockOperation(response)¶

Block execution until API operation is finished.

Parameters

response (Dict) – Google Cloud Build API response.

Returns

Holding the response of a get operation on an API object of type: operations.

Return type

Dict

Raises

RuntimeError – If the Cloud Build failed or if getting the Cloud Build API operation object failed.

CLOUD_BUILD_API_VERSION = 'v1'¶

CreateBuild(build_body)¶

Create a cloud build.

Parameters

build_body (Dict) – A dictionary that describes how to find the source code and how to build it.

Returns

Represents long-running operation that is the result of a network: API call.

Return type

Dict

GcbApi()¶

Get a Google Cloud Build service object.

Returns: A Google Cloud Build service object.
Return type: googleapiclient.discovery.Resource

libcloudforensics.providers.gcp.internal.cloudsql module¶

Google Cloud SQL functionalities.

class libcloudforensics.providers.gcp.internal.cloudsql.GoogleCloudSQL(project_id=None)¶

Bases: object

Class to call Google CloudSQL APIs.

project_id¶: Google Cloud project ID.

GoogleCloudSQLApi()¶

Get a Google CloudSQL service object.

Returns: A Google CloudSQL service object.
Return type: googleapiclient.discovery.Resource

ListCloudSQLInstances()¶

List instances of Google CloudSQL within a project.

Returns: List of instances.
Return type: List[Dict[str, Any]]

SQLADMIN_API_VERSION = 'v1beta4'¶

libcloudforensics.providers.gcp.internal.common module¶

Common utilities.

libcloudforensics.providers.gcp.internal.common.CreateService(service_name, api_version)¶

Creates an GCP API service.

Parameters

service_name (str) – Name of the GCP service to use.
api_version (str) – Version of the GCP service API to use.

Returns

API service resource.

Return type

googleapiclient.discovery.Resource

Raises

CredentialsConfigurationError – If Application Default Credentials could not be obtained
RuntimeError – If service build times out.

libcloudforensics.providers.gcp.internal.common.ExecuteRequest(client, func, kwargs, throttle=False)¶

Execute a request to the GCP API.

Parameters

client (googleapiclient.discovery.Resource) – A GCP client object.
func (str) – A GCP function to query from the client.
kwargs (Dict) – A dictionary of parameters for the function func.
throttle (bool) – A boolean indicating if requests should be throttled. This is necessary for some APIs (e.g. list logs) as there is an API rate limit. Default is False, i.e. requests are not throttled.

Returns

A List of dictionaries (responses from the request).

Return type

List[Dict]

Raises

CredentialsConfigurationError – If the request to the GCP API could not complete.

libcloudforensics.providers.gcp.internal.common.FormatRFC3339(datetime_instance)¶

Formats a datetime per RFC 3339.

Parameters: datetime_instance (datetime) – The datetime group to be formatted.
Returns: A string formatted as per RFC3339 (e.g 2018-05-11T12:34:56.992Z)
Return type: str

libcloudforensics.providers.gcp.internal.common.GenerateDiskName(snapshot, disk_name_prefix=None)¶

Generate a new disk name for the disk to be created from the Snapshot.

The disk name must comply with the following RegEx:

^(?=.{1,63}$)[a-z]([-a-z0-9]*[a-z0-9])?$

i.e., it must be between 1 and 63 chars, the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.

Parameters

snapshot (GoogleComputeSnapshot) – A disk’s Snapshot.
disk_name_prefix (str) – Optional. A prefix for the disk name.

Returns

A name for the disk.

Return type

str

Raises

InvalidNameError – If the disk name does not comply with the RegEx.

libcloudforensics.providers.gcp.internal.common.GenerateSourceRange(exempted_src_ips=None)¶

Generate a list of denied source IP ranges.

The final list is a list of all IPs except the exempted ones.

Parameters: exempted_src_ips (List[str]) – List of IPs exempted from the deny-all ingress firewall rules, ex: analyst IPs.
Returns: Denied source IP ranges specified in CIDR notation.
Return type: List[str]

libcloudforensics.providers.gcp.internal.common.GenerateUniqueInstanceName(prefix, truncate_at=None)¶

Add a timestamp as a suffix to provided name and truncate at max limit.

Parameters

prefix (str) – The name prefix to add the timestamp to and truncate.
truncate_at (int) – Optional. The maximum length of the generated name. Default no limit.

Returns

The name after adding a timestamp.: Ex: [prefix]-[rand]-[TIMESTAMP(‘%Y%m%d%H%M%S’)]

Return type

str

class libcloudforensics.providers.gcp.internal.common.GoogleCloudComputeClient(project_id=None)¶

Bases: object

Class representing Google Cloud Compute API client.

Request and response dictionary content is described here: https://cloud.google.com/compute/docs/reference/rest/v1

project_id¶

Project name.

Type: str

BlockOperation(response, zone=None)¶

Block until API operation is finished.

Parameters

response (Dict) – GCE API response.
zone (str) – Optional. GCP zone to execute the operation in. None means GlobalZone.

Returns

Holding the response of a get operation on an API object of type: zoneOperations or globalOperations.

Return type

Dict

Raises

RuntimeError – If API call failed.

COMPUTE_ENGINE_API_VERSION = 'v1'¶

GceApi()¶

Get a Google Compute Engine service object.

Returns

A Google Compute Engine service: object.

Return type

googleapiclient.discovery.Resource

libcloudforensics.providers.gcp.internal.compute module¶

Google Compute Engine functionalities.

class libcloudforensics.providers.gcp.internal.compute.GoogleCloudCompute(project_id, default_zone=None)¶

Bases: libcloudforensics.providers.gcp.internal.common.GoogleCloudComputeClient

Class representing all Google Cloud Compute objects in a project.

project_id¶: Project name.

default_zone¶: Default zone to create new resources in.

CreateDiskFromImage(src_image, zone, name=None)¶

Creates a GCE persistent disk from a GCE image.

Parameters

src_image (GoogleComputeImage) – Source image for the disk.
zone (str) – Zone to create the new disk in.
name (str) – Optional. Name of the disk to create. Default is [src_image.name]-[TIMESTAMP(‘%Y%m%d%H%M%S’)].

Returns

A Google Compute Disk object.

Return type

GoogleComputeDisk

Raises

InvalidNameError – If GCE disk name is invalid.

CreateDiskFromSnapshot(snapshot, disk_name=None, disk_name_prefix=None, disk_type='pd-standard')¶

Create a new disk based on a Snapshot.

Parameters

snapshot (GoogleComputeSnapshot) – Snapshot to use.
disk_name (str) – Optional. String to use as new disk name.
disk_name_prefix (str) – Optional. String to prefix the disk name with.
disk_type (str) – Optional. URL of the disk type resource describing which disk type to use to create the disk. Default is pd-standard. Use pd-ssd to have a SSD disk. You can list all available disk types by running the following command: gcloud compute disk-types list

Returns

Google Compute Disk.

Return type

GoogleComputeDisk

Raises

ResourceAlreadyExistsError – If the disk already exists.
ResourceCreationError – If the disk could not be created.

CreateImageFromDisk(src_disk, name=None)¶

Creates an image from a persistent disk.

Parameters

src_disk (GoogleComputeDisk) – Source disk for the image.
name (str) – Optional. Name of the image to create. Default is [src_disk.name]-[TIMESTAMP(‘%Y%m%d%H%M%S’)].

Returns

A Google Compute Image object.

Return type

GoogleComputeImage

Raises

InvalidNameError – If the GCE Image name is invalid.
ResourceAlreadyExistsError – If an image with the given name already exists in the project.
ResourceCreationError – If the GCE Image cannot be inserted

CreateImageFromGcsTarGz(gcs_uri, name=None)¶

Creates a GCE image from a Gzip compressed Tar archive in GCS.

Parameters

gcs_uri (str) – Path to the compressed image archive (image.tar.gz) in Cloud Storage. It must be a gzip compressed tar archive with the extension .tar.gz. ex: ‘https://storage.cloud.google.com/foo/bar.tar.gz’ ‘gs://foo/bar.tar.gz’ ‘foo/bar.tar.gz’
name (str) – Optional. Name of the image to create. Default is [src_disk.name]-[TIMESTAMP(‘%Y%m%d%H%M%S’)].

Returns

A Google Compute Image object.

Return type

GoogleComputeImage

Raises

InvalidNameError – If the GCE Image name is invalid.
ValueError – If the extension of the archived image is invalid.

CreateInstanceFromArguments(instance_name, machine_type, zone=None, boot_disk=None, boot_disk_type='pd-standard', boot_disk_size=10, boot_image_project='debian-cloud', boot_image_family='debian-10', additional_scopes=None, sa_email='default', metadata=None, data_disks=None, network_name='default', external_ip=True, owner_label=True)¶

Create a compute instance.

If boot_disk is None then a boot disk need to be created using the ‘boot_disk_size’, ‘cpu_cores’, ‘image_project’ and ‘image_family parameters’.

Parameters

instance_name (str) – Name of the compute instance
machine_type (str) – A string, name of the machine type.
zone (Optional[str]) – Compute zone to start the instance in, default is self.default_zone.
boot_disk (Optional[GoogleComputeDisk]) – Boot disk name.
boot_disk_type (str) – represents persistent disk types, default “pd-standard”.
boot_disk_size (int) – boot disk size in base-2 GB. If you specify a sourceImage, which is required for boot disks, the default size is the size of the sourceImage, else default is 10 GB.
boot_image_project (str) – Name of the project where the boot disk image is stored.
boot_image_family (str) – Name of the image to use to create the boot disk.
additional_scopes (Optional[List[str]]) – additional scopes to be provided to the instance. Default scopes https://cloud.google.com/compute/docs/access/service-accounts#associating_a_service_account_to_an_instance # pylint: disable=line-too-long
sa_email (Optional[str]) – Service account email in case default service account is not used.
metadata (Optional[Dict[str, str]]) – A dictionary mapping metadata keys and values.
data_disks (Optional[List[Union[GoogleComputeDisk, GoogleRegionComputeDisk]]]) – List of disks to attach to the instance, default mode is READ_ONLY.
network_name (str) – Name of the VPC network to use, “default” network is default.
external_ip (bool) – True if the instance should have an external IP.
owner_label (bool) – True if the instance should have an owner label.

Return type

GoogleComputeInstance

Returns

Compute instance object.

Raises

ResourceNotFoundError – If boot disk is not found.
OperationFailedError – If Get operation on boot disk failed.

CreateInstanceFromRequest(request_body, zone=None)¶

Creates an instance from an instance.insert request body.

Parameters

request_body (Dict[str, str]) – Insert instance request body at https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert#request-body # pylint: disable=line-too-long
zone (Optional[str]) – Compute zone to start the instance in, default is self.default_zone.

Return type

GoogleComputeInstance

Returns

Compute instance object.

Raises

ResourceAlreadyExistsError – If an instance with the same name already exists.
ResourceCreationError – If an error occurs creating the instance.
InvalidNameError – If instance name is invalid.

Disks(refresh=True)¶

Get all disks in the project.

Parameters

refresh (boolean) – Optional. Returns refreshed result if True.

Returns

Dictionary mapping disk IDs (str) to: their respective GoogleComputeDisk object.

Return type

Dict[str, GoogleComputeDisk]

GetDisk(disk_name, zone=None)¶

Get a GCP disk object.

Parameters

disk_name (str) – The disk identifier, can be either a disk name or ID.
zone (Optional[str]) – Compute zone.

Returns

Disk object.

Return type

GoogleComputeDisk

Raises

ResourceNotFoundError – When the specified disk cannot be found in project.

GetDiskTypes(disk_type, zone=None)¶

Get selected diskTypes API object in specified zone/project.

Parameters

disk_type (str) – Name of the disk type.
zone (Optional[str]) – Compute zone to get available disk types.

Returns

https://cloud.google.com/compute/docs/reference/rest/v1/diskTypes#resource # pylint: disable=line-too-long

Return type

The selected diskTypes API resource

Raises

HttpError if getting diskType object for the given disk-type name fails. –

GetImageFamily(image_family, project=None)¶

Get image family API object in specified project.

Parameters

image_family (str) – Name of the image famiky.
project (Optional[str]) – Project to get the image family API object from.

Returns

https://cloud.google.com/compute/docs/reference/rest/v1/images/getFromFamily # pylint: disable=line-too-long

Return type

The selected image family API resource

Raises

HttpError if getting image object for the given image family name fails. –

GetInstance(instance_name, zone=None)¶

Get instance from project.

Parameters

instance_name (str) – The instance identifier, can be either an instance name or ID.
zone (Optional[str]) – Compute zone.

Returns

A Google Compute Instance object.

Return type

GoogleComputeInstance

Raises

ResourceNotFoundError – If instance does not exist.

GetMachineTypes(machine_type, zone=None)¶

Get selected machineTypes API object in specified zone/project.

Parameters

machine_type (str) – Name of the machine type.
zone (Optional[str]) – Compute zone to get available machine types.

Returns

https://cloud.google.com/compute/docs/reference/latest/machineTypes#resource # pylint: disable=line-too-long

Return type

The selected machineTypes API resource

Raises

HttpError if getting machineType object for the given machine-type name fails. –

GetNetwork(network_name)¶

Get selected network API object in specified project.

Parameters: network_name (str) – Name of the network.
Returns: https://cloud.google.com/compute/docs/reference/rest/v1/networks # pylint: disable=line-too-long
Return type: The selected network API resource
Raises: HttpError if getting network object for the given machine-type name fails. –

GetOrCreateAnalysisVm(vm_name, boot_disk_size=10, disk_type='pd-standard', cpu_cores=4, image_project='ubuntu-os-cloud', image_family='ubuntu-1804-lts', packages=None, zone=None, data_disks=None)¶

Get or create a new virtual machine for analysis purposes.

If none of the optional parameters are specified, then by default the analysis VM that will be created will run Ubuntu 18.04 LTS. A default set of forensic tools is also installed (a custom one may be provided using the ‘packages’ argument).

Parameters

vm_name (str) – Name of the virtual machine.
boot_disk_size (int) – boot disk size in base-2 GB. If you specify a sourceImage, which is required for boot disks, the default size is the size of the sourceImage, else default is 10 GB.
disk_type (str) – URL of the disk type resource describing which disk type to use to create the disk. Default is pd-standard. Use pd-ssd to have a SSD disk.
cpu_cores (int) – Number of CPU cores for the virtual machine.
image_project (str) – Name of the project where the analysis VM image is hosted.
image_family (str) – Name of the image to use to create the analysis VM.
packages (Optional[List[str]]) – List of packages to install in the VM.
zone (Optional[str]) – Compute zone to start the instance in, default is self.default_zone.
data_disks (Optional[List[Union[GoogleComputeDisk, GoogleRegionComputeDisk]]]) – List of disks to attach to the instance, default mode is READ_ONLY. If the VM already exists, disks will be attached to the existing VM.

Return type

Tuple[GoogleComputeInstance, bool]

Returns

A tuple with a virtual machine object and a boolean indicating: if the virtual machine was created or re-used.

Raises

ResourceCreationError – If virtual machine cannot be found after creation.
ValueError – If the requested CPU cores is not available for the specified machine type.

GetRegionDisk(disk_name, region=None)¶

Get regional disk in project.

Regional disks API resource: https://cloud.google.com/compute/docs/reference/rest/v1/regionDisks#resource:-disk # pylint: disable=line-too-long

Parameters

disk_name (str) – Name or ID of the regional disk to get.
region (Optional[str]) – Compute region.

Return type

GoogleRegionComputeDisk

Returns

Regional disk object.

Raises

ResourceNotFoundError – When the specified disk cannot be found in project.

ImportImageFromStorage(storage_image_path, image_name=None, bootable=False, os_name=None, guest_environment=True)¶

Import GCE image from Cloud storage.

The import tool supports raw disk images and most virtual disk file formats, valid import formats are: [raw (dd), qcow2, qcow , vmdk, vdi, vhd, vhdx, qed, vpc].

Parameters

storage_image_path (str) – Path to the source image in Cloud Storage.
image_name (str) – Optional. Name of the imported image, default is “imported-image-” appended with a timestamp in “%Y%m%d%H%M%S” format.
bootable (bool) – Optional. True if the imported image is bootable. Default is False. If True the os_name must be specified.
os_name (str) – Optional. Name of the operating system on the bootable image. For supported versions please see: https://cloud.google.com/sdk/gcloud/reference/compute/images/import#–os # pylint: disable=line-too-long For known limitations please see: https://googlecloudplatform.github.io/compute-image-tools/image-import.html#compatibility-and-known-limitations # pylint: disable=line-too-long
guest_environment (bool) – Optional. Install Google Guest Environment on a bootable image. Relevant only if image is bootable. Default True.

Returns

A Google Compute Image object.

Return type

GoogleComputeImage

Raises

ValueError – If bootable is True and os_name not specified.
InvalidNameError – If imported image name is invalid.

InsertFirewallRule(body)¶

Insert a firewall rule to the project.

Parameters: body (Dict) – The request body. https://googleapis.github.io/google-api-python-client/docs/dyn/compute_v1.firewalls.html#insert # pylint: disable=line-too-long
Return type: None

Instances(refresh=True)¶

Get all instances in the project.

Parameters

refresh (boolean) – Optional. Returns refreshed result if True.

Returns

Dictionary mapping instance IDs: (str) to their respective GoogleComputeInstance object.

Return type

Dict[str, GoogleComputeInstance]

ListComputeRegions()¶

List Compute regions in the project

Return type: List[str]
Returns: List of all regions.

ListDiskByLabels(labels_filter, filter_union=True)¶

List Disks in a project with one/all of the provided labels.

This will call the _ListByLabel function on a disks() API object with the proper labels filter and return a Dict with name and metadata for each disk, e.g.:

{‘disk-1’: {‘zone’: ‘us-central1-a’, ‘labels’: {‘id’: ‘123’}}

Parameters

labels_filter (Dict[str, str]) – A Dict of labels to find e.g. {‘id’: ‘123’}.
filter_union (bool) – Optional. A Boolean; True to get the union of all filters, False to get the intersection.

Returns

Dictionary mapping disks to their: respective GoogleComputeDisk object.

Return type

Dict[str, GoogleComputeDisk]

ListDisks()¶

List disks in project.

Returns

Dictionary mapping disk names (str) to: their respective GoogleComputeDisk object.

Return type

Dict[str, GoogleComputeDisk]

ListInstanceByLabels(labels_filter, filter_union=True)¶

List VMs in a project with one/all of the provided labels.

This will call the _ListByLabel function on an instances() API object with the proper labels filter and return a Dict with name and metadata for each instance, e.g.:

{‘instance-1’: {‘zone’: ‘us-central1-a’, ‘labels’: {‘id’: ‘123’}}

Parameters

labels_filter (Dict[str, str]) – A Dict of labels to find e.g. {‘id’: ‘123’}.
filter_union (bool) – Optional. A Boolean; True to get the union of all filters, False to get the intersection.

Returns

Dictionary mapping instances to their: respective GoogleComputeInstance object.

Return type

Dict[str, GoogleComputeInstance]

ListInstances()¶

List instances in project.

Returns

Dictionary mapping instance IDs (str): to their respective GoogleComputeInstance object.

Return type

Dict[str, GoogleComputeInstance]

ListMIGS(zone)¶

Gets the managed instance groups in a particular zone.

Returns a dictionary, with as keys the managed instance groups, and as values a list of instances belonging to the group.

Parameters

zone (str) – The zone in which to list managed instance groups.

Returns

A mapping from managed instance: groups to their managed GCE instances.

Return type

Dict[str, List[GoogleComputeInstance]]

ListMIGSByInstanceName(zone)¶

Gets a mapping from instance names to their managed instance group.

Parameters

zone (str) – The zone in which to list managed instance groups.

Returns

A mapping from instance names to their managed instance: group.

Return type

Dict[str, str]

Raises

RuntimeError – If multiple managed instance groups are found for a single instance.

ListRegionDisks()¶

List regional disks in project.

Return type

Dict[str, GoogleRegionComputeDisk]

Returns

Dictionary mapping disk names (str) to: their respective GoogleRegionComputeDisk object.

ListReservedExternalIps(zone)¶

Lists all static external IP addresses that are available to a zone.

The method first converts the zone to a region, and then queries the GCE addresses resource.

Parameters

zone (str) – The zone in which the returned IPs would be available.

Returns

The list of available IPs in the specified zone.

Return type

List[str]

Raises

ValueError – If the zone is malformed.
errors.ResourceNotFoundError – If the request did not succeed.

RegionDisks(refresh=True)¶

Get all regional disks in the project.

Parameters

refresh (Optional[bool]) – Returns refreshed result if True.

Return type

Dict[str, GoogleRegionComputeDisk]

Returns

Dictionary mapping disk IDs (str) to their respective: GoogleRegionComputeDisk object.

class libcloudforensics.providers.gcp.internal.compute.GoogleComputeDisk(project_id, zone, name, resource_id=None, labels=None, deletion_protection=False, region=None)¶

Bases: libcloudforensics.providers.gcp.internal.compute_base_resource.GoogleComputeBaseResource

Class representing a Compute Engine disk.

Delete()¶

Delete a Disk.

Return type: None

GetDiskType()¶

Return the disk type.

Returns: The disk type.
Return type: str

GetOperation()¶

Get API operation object for the disk.

Returns

An API operation object for a Google Compute Engine disk.: https://cloud.google.com/compute/docs/reference/rest/v1/disks/get#response-body

Return type

Dict

Snapshot(snapshot_name=None)¶

Create Snapshot of the disk.

The Snapshot name must comply with the following RegEx:

^(?=.{1,63}$)[a-z]([-a-z0-9]*[a-z0-9])?$

i.e., it must be between 1 and 63 chars, the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.

Parameters

snapshot_name (str) – Optional. Name of the Snapshot.

Returns

A Snapshot object. created: False if the resource existed already.

Return type

GoogleComputeSnapshot

Raises

InvalidNameError – If the name of the snapshot does not comply with the RegEx.
RuntimeError – If the snapshot operation fails.

class libcloudforensics.providers.gcp.internal.compute.GoogleComputeImage(project_id, zone, name, resource_id=None, labels=None, deletion_protection=False, region=None)¶

Bases: libcloudforensics.providers.gcp.internal.compute_base_resource.GoogleComputeBaseResource

Class representing a Compute Engine Image.

Delete()¶

Delete Compute Disk Image from a project.

Return type: None

ExportImage(gcs_output_folder, output_name=None)¶

Export compute image to Google Cloud storage.

Exported image is compressed and stored in .tar.gz format.

Parameters

gcs_output_folder (str) – Folder path of the exported image.
output_name (str) – Optional. Name of the output file. Name will be appended with .tar.gz. Default is [image_name].tar.gz.

Raises

InvalidNameError – If exported image name is invalid.

Return type

None

GetOperation()¶

Get API operation object for the image.

Returns

Holding an API operation object for a Google Compute Engine Image.: https://cloud.google.com/compute/docs/reference/rest/v1/images/get#response-body

Return type

Dict

class libcloudforensics.providers.gcp.internal.compute.GoogleComputeInstance(project_id, zone, name, resource_id=None, labels=None, deletion_protection=False, region=None)¶

Bases: libcloudforensics.providers.gcp.internal.compute_base_resource.GoogleComputeBaseResource

Class representing a Google Compute Engine virtual machine.

AbandonFromMIG(instance_group)¶

Abandons the instance from the managed instance group.

Parameters: instance_group (str) – The instance group that this instance should be abandoned from.
Raises: errors.OperationFailedError – If the request did not succeed.
Return type: None

AssignExternalIp(net_if, ip_addr=None)¶

Assigns an external IP to an instance’s network interface.

The instance must not have an IP assigned to the network interface when calling this method. If the IP address is specified, it must be one that is available to the project.

Parameters

net_if (str) – The instance’s network interface to which the IP address must be assigned.
ip_addr (str) – Optional. The static IP address that exposes the network interface. If None, the assigned IP address will be ephemeral.

Raises

errors.ResourceCreationError – If the assignment did not succeed.

Return type

None

AttachDisk(disk, read_write=False)¶

Attach a disk to the virtual machine.

Parameters

disk (Union[GoogleComputeDisk, GoogleRegionComputeDisk]) – Disk to attach.
read_write (bool) – Boolean indicating whether the disk should be attached in RW mode. Default is False (read-only).

Return type

None

Delete(delete_disks=False, force_delete=False)¶

Delete an Instance.

Parameters

delete_disks (bool) – force delete all attached disks (ignores the ‘Keep when instance is deleted’ bit).
force_delete (bool) – force delete the instance, even if deletionProtection is set to true.

Raises

ResourceDeletionError – If deleteProtection could not be toggled on the instance

Return type

None

DetachDisk(disk)¶

Detach a disk from the virtual machine.

Parameters: disk (GoogleComputeDisk) – Disk to detach.
Return type: None

DetachServiceAccount()¶

Detach a service account from the instance

Raises: errors.ServiceAccountRemovalError – if en error occurs while detaching the service account
Return type: None

GetBootDisk()¶

Get the virtual machine boot disk.

Returns: Disk object.
Return type: GoogleComputeDisk
Raises: ResourceNotFoundError – If no boot disk could be found.

GetDisk(disk_name)¶

Gets a disk attached to this virtual machine disk by name.

Parameters: disk_name (str) – The name of the disk to get.
Returns: Disk object.
Return type: GoogleComputeDisk
Raises: ResourceNotFoundError – If disk name is not found among those attached to the instance.

GetEffectiveFirewalls()¶

Get the raw effective firewalls for every interface: https://cloud.google.com/compute/docs/reference/rest/v1/instances/getEffectiveFirewalls

Returns: The effective firewall rules per interface.
Return type: List[Dict[str, Any]]

GetNatIps()¶

Get the NAT external IPv4 addresses attached to an instance.

Returns: a list of IP addresses.
Return type: List[str]

GetNormalisedFirewalls()¶

Get normalised effective firewalls for every interface with firewall policies and normal VPC firewalls collapsed into a single list.

Returns: The normalised firewalls per interface.
Return type: List[Dict[str, Any]]

GetOperation()¶

Get API operation object for the virtual machine.

Returns

An API operation object for a Google Compute Engine: virtual machine. https://cloud.google.com/compute/docs/reference/rest/v1/instances/get#response-body

Return type

Dict

GetPowerState()¶

Gets the current power state of the instance.

As per https://cloud.google.com/compute/docs/reference/rest/v1/instances/get this can return one of the following possible values: PROVISIONING, STAGING, RUNNING, STOPPING, SUSPENDING, SUSPENDED, REPAIRING, and TERMINATED

Return type: str

ListDisks()¶

List all disks for the virtual machine.

Returns

Dictionary mapping disk names to their: respective GoogleComputeDisk object.

Return type

Dict[str, GoogleComputeDisk]

RemoveExternalIps()¶

Removes any external IP of the instance, breaking ongoing connections.

Note that if the instance’s IP address was static, that the IP will still belong to the project.

Returns

A mapping from an instance’s network: interfaces to the corresponding removed external IP.

Return type

Dict[str, str]

Raises

errors.ResourceDeletionError – If the removal did not succeed.

SetTags(new_tags)¶

Sets tags for the compute instance.

Tags are used to configure firewall rules and network routes.

Parameters: new_tags (List[str]) – A list of tags. Each tag must be 1-63 characters long, and comply with RFC1035.
Raises: InvalidNameError – If the name of the tags does not comply with RFC1035.
Return type: None

Ssh()¶

Connect to the virtual machine over SSH.

Return type: None

Start()¶

Starts the instance.

Raises: errors.InstanceStateChangeError – If the Start operation is unsuccessful
Return type: None

Stop()¶

Stops the instance.

Raises: errors.InstanceStateChangeError – If the Stop operation is unsuccessful
Return type: None

class libcloudforensics.providers.gcp.internal.compute.GoogleComputeSnapshot(disk, name)¶

Bases: libcloudforensics.providers.gcp.internal.compute_base_resource.GoogleComputeBaseResource

Class representing a Compute Engine Snapshot.

disk¶

Disk used for the Snapshot.

Type: GoogleComputeDisk

Delete()¶

Delete a Snapshot.

Return type: None

GetOperation()¶

Get API operation object for the Snapshot.

Returns

An API operation object for a Google Compute Engine Snapshot.: https://cloud.google.com/compute/docs/reference/rest/v1/snapshots/get#response-body

Return type

Dict

class libcloudforensics.providers.gcp.internal.compute.GoogleRegionComputeDisk(project_id, region, name, **kwargs)¶

Bases: libcloudforensics.providers.gcp.internal.compute_base_resource.GoogleComputeBaseResource

Class representing a regional compute disk.

project_id¶: Project ID.

region¶: Compute region in which the disk resides.

name¶: name of the regional compute disk.

GetOperation()¶

Get API operation object for the regional disk.

Return type

Dict[str, Any]

Returns

An API operation object for a Google Regional Compute Engine disk.: https://cloud.google.com/compute/docs/reference/rest/v1/regionDisks/get#response-body # pylint: disable=line-too-long

libcloudforensics.providers.gcp.internal.compute_base_resource module¶

Google Compute Engine resources.

class libcloudforensics.providers.gcp.internal.compute_base_resource.GoogleComputeBaseResource(project_id, zone, name, resource_id=None, labels=None, deletion_protection=False, region=None)¶

Bases: libcloudforensics.providers.gcp.internal.common.GoogleCloudComputeClient

Base class representing a Computer Engine resource.

project_id¶

Google Cloud project ID.

Type: str

zone¶

What zone the resource is in.

Type: str

name¶

Name of the resource.

Type: str

resource_id¶

The ID number of the resource.

Type: str

labels¶

Dictionary of labels for the resource, if existing.

Type: Dict

deletion_protection¶

True if the resource has deletionProtection enabled.

Type: bool

AddLabels(new_labels_dict, blocking_call=False)¶

Add or update labels of a compute resource.

Parameters

new_labels_dict (Dict) – A dictionary containing the labels to be added, ex:{“incident_id”: “1234abcd”}.
blocking_call (bool) – Optional. A boolean to decide whether the API call should be blocking or not, default is False.

Returns

The response of the API operation (a Dict if the call is: successful).

Return type

Optional[Any]

Raises

RuntimeError – If the Compute resource Type is not one of instance, disk or snapshot.

FormOperation(operation_name)¶

Form an API operation object for the compute resource.

Example:[RESOURCE].FormOperation(‘setLabels’)(**kwargs) [RESOURCE] can be type “instance”, disk or “Snapshot”.

Parameters

operation_name (str) – The name of the API operation you need to perform.

Returns

An API operation object for the: referenced compute resource.

Return type

googleapiclient.discovery.Resource

Raises

RuntimeError – If resource type is not defined as a type which extends the GoogleComputeBaseResource class.

FormatLogMessage(message)¶

Format log messages with project specific information.

Parameters: message (str) – Message string to log.
Returns: Formatted log message string.
Return type: str

GetLabels()¶

Get all labels of a compute resource.

Returns: A dictionary of all labels.
Return type: Dict

GetOperation()¶

Abstract method to be implemented by child classes.

Raises: NotImplementedError – If the child class doesn’t implement GetOperation.
Return type: Dict[str, Any]

GetResourceType()¶

Get the resource type from the resource key-value store.

Returns

Resource Type which is a string with one of the following values:: compute#instance compute#disk compute#Snapshot

Return type

str

GetSourceString()¶

API URL to the resource.

Returns: The full API URL to the resource.
Return type: str

GetValue(key)¶

Get specific value from the resource key value store.

Parameters: key (str) – A key of type String to get key’s corresponding value.
Returns: Value of key/dictionary or None if key is missing.
Return type: str|Dict

libcloudforensics.providers.gcp.internal.function module¶

Google Cloud Functions functionalities.

class libcloudforensics.providers.gcp.internal.function.GoogleCloudFunction(project_id)¶

Bases: object

Class to call Google Cloud Functions.

project_id¶: Google Cloud project ID.

CLOUD_FUNCTIONS_API_VERSION = 'v1'¶

ExecuteFunction(function_name, region, args)¶

Executes a Google Cloud Function.

Parameters

function_name (str) – The name of the function to call.
region (str) – Region to execute functions in.
args (Dict) – Arguments to pass to the function. Dictionary content details can be found in https://cloud.google.com/functions/docs/reference/rest/v1/projects.locations.functions # pylint: disable=line-too-long

Returns

Return value from function call.

Return type

Dict[str, str]

Raises

RuntimeError – When cloud function arguments cannot be serialized or when an HttpError is encountered.

GcfApi()¶

Get a Google Cloud Function service object.

Returns

A Google Cloud Function service: object.

Return type

googleapiclient.discovery.Resource

libcloudforensics.providers.gcp.internal.gke module¶

Google Kubernetes Engine functionalities.

class libcloudforensics.providers.gcp.internal.gke.GkeCluster(project_id, zone, cluster_id)¶

Bases: libcloudforensics.providers.kubernetes.cluster.K8sCluster, libcloudforensics.providers.gcp.internal.gke.GoogleKubernetesEngine

Class to call GKE and Kubernetes APIs on a GKE resource.

https://cloud.google.com/kubernetes-engine/docs/reference/rest https://kubernetes.io/docs/reference/

project_id¶

The GCP project name.

Type: str

zone¶

The GCP zone for this project.

Type: str

cluster_id¶

The name of the GKE cluster.

Type: str

ClusterLogsQuery(workload=None)¶

Creates the GCP k8s_cluster logs query string for this cluster.

A workload may optionally be specified, in which case the returned query string will be more specific to only cover that workload.

Parameters: workload (base.K8sWorkload) – Optional. A workload to specify in the query string.
Returns: The k8s_cluster logs query string.
Return type: str

ContainerLogsQuery(workload=None)¶

Returns the GCP k8s_container logs query string for this cluster.

A workload may optionally be specified, in which case the returned query string will be more specific to only cover that workload.

Parameters: workload (base.K8sWorkload) – Optional. A workload to specify in the query string.
Returns: The k8s_container logs query string.
Return type: str

GetOperation()¶

Get GKE API operation object for the GKE resource.

Returns: GKE API response to ‘get’ operation for this cluster.
Return type: Dict[str, Any]

IsLegacyEndpointsDisabled()¶

Returns whether legacy endpoints are enabled.

Returns: True if legacy endpoints are disabled, False otherwise.
Return type: bool

IsNetworkPolicyEnabled()¶

Override of abstract method.

Return type: bool

IsWorkloadIdentityEnabled()¶

Returns whether the workload identity is enabled.

Returns: True if workload identity is enabled, False otherwise.
Return type: bool

property name¶

Name of the GKE cluster resource, for use in API calls.

Returns: Full name of the cluster resource.
Return type: str

class libcloudforensics.providers.gcp.internal.gke.GoogleKubernetesEngine¶

Bases: object

Base class for calling GKE APIs.

GKE_API_VERSION = 'v1'¶

GkeApi()¶

Gets a Google Container service object.

https://container.googleapis.com/$discovery/rest?version=v1

Returns: A Google Container service object.
Return type: googleapiclient.discovery.Resource

libcloudforensics.providers.gcp.internal.log module¶

Google Cloud Logging functionalities.

class libcloudforensics.providers.gcp.internal.log.GoogleCloudLog(project_ids)¶

Bases: object

Class representing a Google Cloud Logs interface.

project_ids¶: List of Google Cloud project IDs.

Example use:: # pylint: disable=line-too-long gcp = GoogleCloudLog(project_id=’your_project_name’) gcp.ListLogs() gcp.ExecuteQuery(filter=’resource.type=”gce_instance” labels.”compute.googleapis.com/resource_name”=”instance-1”’) See https://cloud.google.com/logging/docs/view/advanced-queries for filter details.

ExecuteQuery(qfilter=None)¶

Query logs in GCP project.

Parameters

qfilter (List[str]) – Optional. A list of query filters to use.

Returns

Log entries returned by the query, e.g. [{‘projectIds’:: […], ‘resourceNames’: […]}, {…}]

Return type

List[Dict]

Raises

RuntimeError – If API call failed.
ValueError – If the number of project IDs being queried doesn’t match the number of provided filters.

GclApi()¶

Get a Google Compute Logging service object.

Returns

A Google Compute Logging service: object.

Return type

googleapiclient.discovery.Resource

LOGGING_API_VERSION = 'v2'¶

ListLogs()¶

List logs in project.

Returns: The project logs available.
Return type: List[str]
Raises: RuntimeError – If API call failed.

libcloudforensics.providers.gcp.internal.monitoring module¶

Google Cloud Monitoring functionality.

class libcloudforensics.providers.gcp.internal.monitoring.GoogleCloudMonitoring(project_id)¶

Bases: object

Class to call Google Monitoring APIs.

https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.timeSeries

project_id¶: Project name.

ActiveServices(timeframe=30)¶

List active services in the project (default: last 30 days).

Parameters: timeframe (int) – Optional. The number (in days) for which to measure activity.
Returns: Dictionary mapping service name to number of uses.
Return type: Dict[str, int]

CLOUD_MONITORING_API_VERSION = 'v3'¶

GcmApi()¶

Get a Google Cloud Monitoring service object.

Returns

A Google Cloud Monitoring: service object.

Return type

googleapiclient.discovery.Resource

GetCpuUsage(instance_ids=None, days=7, aggregation_minutes=60)¶

Returns CPU usage metrics for compute instances.

By default returns hourly usage for the last seven days for all instances within a project.

Parameters

list[str] (instance_ids) – Optional. A list of instance IDs to collect metrics for. When not provided will collect metrics for all instances in the project.
days (int) – Optional. The number of days to collect metrics for.
aggregation_minutes (int) – Optional. The minutes to aggregate on.

Returns

a list of CPU usage for each instance in the format

[

{

‘instance_name’: str, ‘instance_id’: str, ‘cpu_usage’: [

{
‘timestamp’: str, ‘cpu_usage’: float

},

]

},

]

Return type

List[Dict[str, Any]]

libcloudforensics.providers.gcp.internal.project module¶

Google Cloud Project resources and services.

class libcloudforensics.providers.gcp.internal.project.GoogleCloudProject(project_id=None, default_zone='us-central1-f')¶

Bases: object

Class representing a Google Cloud Project.

project_id¶: Google Cloud project ID.

default_zone¶: Default zone to create new resources in.

Example use:: gcp = GoogleCloudProject(“your_project_name”, “us-east1-b”) gcp.compute.ListInstances()

Delete()¶

Delete a GCP project.

Returns: The operation’s result details.
Return type: Dict[str, Any]

property bigquery¶

Get a GoogleBigQuery object for the project.

Returns: Object that represents Google BigQuery.
Return type: GoogleBigQuery

property build¶

Get a GoogleCloudBuild object for the project.

Returns: Object that represents Google Cloud Build.
Return type: GoogleCloudBuild

property cloudresourcemanager¶

Get a GoogleCloudResourceManager object for the project.

Returns

Object that represents Google cloud resource: manager.

Return type

GoogleCloudResourceManager

property cloudsql¶

Get a GoogleCloudSql object for the project.

Returns: Object that represents Google SQL.
Return type: GoogleCloudSql

property compute¶

Get a GoogleCloudCompute object for the project.

Returns: Object that represents Google Cloud Compute Engine.
Return type: GoogleCloudCompute

property function¶

Get a GoogleCloudFunction object for the project.

Returns: Object that represents Google Cloud Function.
Return type: GoogleCloudFunction

property gke¶

Get a GoogleKubernetesEngine object for the project.

Returns: Object that represents Google Kubernetes Engine.
Return type: GoogleKubernetesEngine

property log¶

Get a GoogleCloudLog object for the project.

Returns: Object that represents Google Cloud Logging.
Return type: GoogleCloudLog

property monitoring¶

Get a GoogleCloudMonitoring object for the project.

Returns: Object that represents Google Monitoring.
Return type: GoogleCloudMonitoring

property serviceusage¶

Get a GoogleServiceUsage object for the project.

Returns: Object that represents Google service usage.
Return type: GoogleServiceUsage

property storage¶

Get a GoogleCloudStorage object for the project.

Returns: Object that represents Google Cloud Storage.
Return type: GoogleCloudStorage

property storagetransfer¶

Get a GoogleCloudStorageTransfer object for the project.

Returns: Object that represents Google Cloud Storage Transfer.
Return type: GoogleCloudStorageTransfer

libcloudforensics.providers.gcp.internal.storage module¶

Google Cloud Storage functionalities.

class libcloudforensics.providers.gcp.internal.storage.GoogleCloudStorage(project_id=None)¶

Bases: object

Class to call Google Cloud Storage APIs.

project_id¶: Google Cloud project ID.

CLOUD_STORAGE_API_VERSION = 'v1'¶

CreateBucket(bucket, labels=None, predefined_acl='private', predefined_default_object_acl='private')¶

Creates a Google Cloud Storage bucket in the current project.

Parameters

bucket (str) – Name of the desired bucket.
labels (Dict[str, str]) – Mapping of key/value strings to be applied as a label to the bucket. Rules for acceptable label values are located at https://cloud.google.com/storage/docs/key-terms#bucket-labels
predefined_acl (str) – A predefined set of Access Controls to apply to the bucket.
predefined_default_object_acl (str) – A predefined set of Access Controls to apply to the objects in the bucket.
listed in https (Values) – //cloud.google.com/storage/docs/json_api/v1/buckets/insert#parameters # pylint: disable=line-too-long

Returns

An API operation object for a Google Cloud Storage bucket.: https://cloud.google.com/storage/docs/json_api/v1/buckets#resource

Return type

Dict[str, Any]

Raises

ResourceCreationError – If the resource could not be created.

DeleteObject(gcs_path)¶

Deletes an object in a Google Cloud Storage bucket.

Parameters: gcs_path (str) – Full path to the object (ie: gs://bucket/dir1/dir2/obj)
Return type: None

GcsApi()¶

Get a Google Cloud Storage service object.

Returns: A Google Cloud Storage service object.
Return type: googleapiclient.discovery.Resource

GetBucketACLs(bucket, user_project=None)¶

Get ACLs for a Google Cloud Storage bucket.

This includes both ACL entries and IAM policies.

Parameters

bucket (str) – Name of a bucket in GCS. Ex: logs_bucket_1
user_project (str) – The project ID to be billed for this request. Required for Requester Pays buckets.

Returns

A mapping of role to members of that role.

Return type

Dict

GetBucketSize(bucket, timeframe=1)¶

List the size of a Google Storage Bucket in a project (default: last 1 day).

Note: This will list the _maximum size_: (in bytes) the bucket had in the timeframe.

Ref: https://cloud.google.com/monitoring/api/metrics_gcp#gcp-storage

Parameters

bucket (str) – Name of a bucket in GCS.
timeframe (int) – Optional. The number (in days) for which to measure activity. Default: 1 day.

Returns

Dictionary mapping bucket name to its size (in bytes).

Return type

Dict[str, int]

GetObject(gcs_path, out_file=None)¶

Gets the contents of an object in a Google Cloud Storage bucket.

Parameters

gcs_path (str) – Full path to the object (ie: gs://bucket/dir1/dir2/obj)
out_file (str) – Path to the local file that will be written. If not provided, will create a temporary file.

Returns

The filename of the written object.

Return type

str

Raises

ResourceCreationError – If the file couldn’t be downloaded.

GetObjectMetadata(gcs_path, user_project=None)¶

Get API operation object metadata for Google Cloud Storage object.

Parameters

gcs_path (str) – File path to a resource in GCS. Ex: gs://bucket/folder/obj
user_project (str) – The project ID to be billed for this request. Required for Requester Pays buckets.

Returns

An API operation object for a Google Cloud Storage object.: https://cloud.google.com/storage/docs/json_api/v1/objects#resource

Return type

Dict

ListBucketObjects(bucket)¶

List objects (with metadata) in a Google Cloud Storage bucket.

Parameters: bucket (str) – Name of a bucket in GCS.
Return type: List[Dict[str, Any]]
Returns: List of Object Dicts (see GetObjectMetadata)

ListBuckets()¶

List buckets in a Google Cloud project.

Returns: List of object dicts. (https://cloud.google.com/storage/docs/json_api/v1/buckets#resource)
Return type: List[Dict[str, Any]]