| # -*- coding: utf-8 -*- |
| # Copyright 2015 Google Inc. All Rights Reserved. |
| # |
| # Licensed under the Apache License, Version 2.0 (the "License"); |
| # you may not use this file except in compliance with the License. |
| # You may obtain a copy of the License at |
| # |
| # http://www.apache.org/licenses/LICENSE-2.0 |
| # |
| # Unless required by applicable law or agreed to in writing, software |
| # distributed under the License is distributed on an "AS IS" BASIS, |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| # See the License for the specific language governing permissions and |
| # limitations under the License. |
| """Additional help about types of credentials and authentication.""" |
| |
| from __future__ import absolute_import |
| |
| from gslib.help_provider import HelpProvider |
| |
| _DETAILED_HELP_TEXT = (""" |
| <B>OVERVIEW</B> |
| This help section provides details about various precautions taken by gsutil |
| to protect data security, as well as recommendations for how customers should |
| safeguard security. |
| |
| |
| <B>TRANSPORT LAYER SECURITY</B> |
| gsutil performs all operations using transport-layer encryption (HTTPS), to |
| protect against data leakage over shared network links. This is also important |
| because gsutil uses "bearer tokens" for authentication (OAuth2) as well as for |
| resumable upload identifiers, and such tokens must be protected from being |
| eavesdropped and reused. |
| |
| gsutil also supports the older HMAC style of authentication via the XML API |
| (see "gsutil help apis"). While HMAC authentication does not use bearer |
| tokens (and thus is not subject to eavesdropping/replay attacks), it's still |
| important to encrypt data traffic. |
| |
| Prior to gsutil release 4.0 it was possible to use HTTP instead of HTTPS by |
| setting the "is_secure" configuration parameter in the [Boto] section of the |
| boto configuration file to False. However, starting with gsutil version 4.0 |
| setting is_secure to False is disallowed. For more details about different |
| credential options, see "gsutil help creds". |
| |
| |
| <B>LOCAL FILE STORAGE SECURITY</B> |
| gsutil takes a number of precautions to protect against security exploits in |
| the files it stores locally: |
| |
| - When the gsutil config command runs it sets file protection mode 600 |
| ("-rw-------") on the the .boto configuration file it generates, so only |
| the user (or superuser) can read it. This is important because these files |
| contain security-sensitive information, including credentials and proxy |
| configuration. |
| |
| - The gsutil config command also uses file protection mode 600 for the |
| private key file stored locally when you create service account |
| credentials. |
| |
| - The default level of logging output from gsutil commands does not include |
| security-sensitive information, such as OAuth2 tokens and proxy |
| configuration information. (See the "RECOMMENDED USER PRECAUTIONS" section |
| below if you increase the level of debug output, using the gsutil -D |
| option.) |
| |
| Note that protection modes are not supported on Windows, so if you |
| use gsutil on Windows we recommend using an encrypted file system and strong |
| account passwords. |
| |
| |
| <B>SECURITY-SENSITIVE FILES WRITTEN TEMPORARILY TO DISK BY GSUTIL</B> |
| gsutil buffers data in temporary files in several situations: |
| |
| - While compressing data being uploaded via gsutil cp -z, gsutil |
| buffers the data in temporary files with protection 600, which it |
| deletes after the upload is complete (similarly for downloading files |
| that were uploaded with gsutil cp -z or some other process that sets the |
| Content-Encoding to "gzip"). However, if you kill the gsutil process |
| while the upload is under way the partially written file will be left |
| in place. See the "CHANGING TEMP DIRECTORIES" section in |
| "gsutil help cp" for details of where the temporary files are written |
| and how to change the temp directory location. |
| |
| - When performing a resumable upload gsutil stores the upload ID (which, |
| as noted above, is a bearer token and thus should be safe-guarded) in a |
| file under ~/.gsutil/tracker-files with protection 600, and deletes this |
| file after the upload is complete. However, if the upload doesn't |
| complete successfully the tracker file is left in place so the resumable |
| upload can be re-attempted later. Over time it's possible to accumulate |
| these tracker files from aborted upload attempts, though resumable |
| upload IDs are only valid for 1 week, so the security risk only exists |
| for files less than that old. If you consider the risk of leaving |
| aborted upload IDs in the tracker directory too high you could modify |
| your upload scripts to delete the tracker files; or you could create a |
| cron job to clear the tracker directory periodically. |
| |
| - The gsutil rsync command stores temporary files (with protection 600) |
| containing the names, sizes, and checksums of source and destination |
| directories/buckets, which it deletes after the rsync is complete. |
| However, if you kill the gsutil process while the rsync is under way the |
| listing files will be left in place. |
| |
| Note that gsutil deletes temporary files using the standard OS unlink system |
| call, which does not perform `data wiping |
| <https://en.wikipedia.org/wiki/Data_erasure>`_. Thus, the content of such |
| temporary files can be recovered by a determined adversary. |
| |
| |
| <B>ACCESS CONTROL LISTS</B> |
| Unless you specify a different ACL (e.g., via the gsutil cp -a option), by |
| default objects written to a bucket use the default object ACL on that bucket. |
| Unless you modify that ACL (e.g., via the gsutil defacl command), by default |
| it will allow all project editors write access to the object and read/write |
| access to the object's metadata; and will allow all project viewers read |
| access to the object. |
| |
| The GCS access control system includes the ability to specify that objects are |
| publicly readable. Make sure you intend for any objects you write with this |
| permission to be public. Once "published", data on the Internet can be copied |
| to many places, so it's effectively impossible to regain read control over an |
| object written with this permission. |
| |
| The GCS access control system includes the ability to specify that buckets are |
| publicly writable. While configuring a bucket this way can be convenient for |
| various purposes, we recommend against using this permission - it can be |
| abused for distributing illegal content, viruses, and other malware, and the |
| bucket owner is legally and financially responsible for the content stored in |
| their buckets. If you need to make content available to customers who don't |
| have Google accounts consider instead using signed URLs (see |
| "gsutil help signurl"). |
| |
| |
| <B>SOFTWARE INTEGRITY AND UPDATES</B> |
| gsutil is distributed as a standalone bundle via tar and zip files stored in |
| the gs://pub bucket, as a PyPi module, and as part of the bundled Cloud |
| SDK release. Each of these distribution methods takes a variety of security |
| precautions to protect the integrity of the software. We strongly recommend |
| against getting a copy of gsutil from any other sources (such as mirror |
| sites). |
| |
| |
| <B>PROXY USAGE</B> |
| gsutil supports access via proxies, such as Squid and a number of commercial |
| products. A full description of their capabilities is beyond the scope of this |
| documentation, but proxies can be configured to support many security-related |
| functions, including virus scanning, Data Leakage Prevention, control over |
| which certificates/CA's are trusted, content type filtering, and many more |
| capabilities. Some of these features can slow or block legitimate gsutil |
| behavior. For example, virus scanning depends on decrypting file content, |
| which in turn requires that the proxy terminate the gsutil connection and |
| establish a new connection - and in some cases proxies will rewrite content in |
| ways that result in checksum validation errors and other problems. |
| |
| For details on configuring proxies see the proxy help text in your .boto |
| configuration file (generated by the gsutil config command). |
| |
| |
| <B>ENCRYPTION AT REST</B> |
| All GCS data are stored encrypted. For more information see |
| `Server-Side Encryption |
| <https://cloud.google.com/storage/docs/concepts-techniques#encryption>`_. |
| |
| |
| <B>DATA PRIVACY FROM GOOGLE EMPLOYEES</B> |
| Google employees will never look at your data unless you first explicitly |
| grant them permission to do so while troubleshooting a specific incident. |
| |
| Google will never ask you to share your credentials, password, or other |
| security-sensitive information. Beware of potential phishing scams where |
| someone attempts to impersonate Google and asks for such information. |
| |
| |
| <B>MEASUREMENT DATA</B> |
| The gsutil perfdiag command collects a variety of performance-related |
| measurements and details about your local system and network environment, for |
| use in troubleshooting performance problems. None of this information will be |
| sent to Google unless you choose to send it. |
| |
| |
| <B>RECOMMENDED USER PRECAUTIONS</B> |
| The first and foremost precaution is: Never share your credentials. Each user |
| should have distinct credentials. |
| |
| If you run gsutil -D (to generate debugging output) it will include OAuth2 |
| refresh and access tokens in the output. Make sure to redact this information |
| before sending this debug output to anyone during troubleshooting/tech support |
| interactions. |
| |
| If you run gsutil --trace-token (to send a trace directly to Google), |
| sensitive information like OAuth2 tokens and the contents of any files |
| accessed during the trace may be included in the content of the trace. |
| |
| The proxy configuration information in the .boto configuration is |
| security-sensitive, especially if your proxy setup requires user and |
| password information. Even if your proxy setup doesn't require user and |
| password, the host and port number for your proxy is often considered |
| security-sensitive. Protect access to your .boto configuration file. |
| |
| If you are using gsutil from a production environment (e.g., via a cron job |
| running on a host in your data center), use service account credentials rather |
| than individual user account credentials. These credentials were designed for |
| such use and, for example, protect you from losing access when an employee |
| leaves your company. |
| """) |
| |
| |
| class CommandOptions(HelpProvider): |
| """Additional help about security and privacy considerations using gsutil.""" |
| |
| # Help specification. See help_provider.py for documentation. |
| help_spec = HelpProvider.HelpSpec( |
| help_name='security', |
| help_name_aliases=['encryption', 'protection', 'privacy', 'proxies', |
| 'proxy'], |
| help_type='additional_help', |
| help_one_line_summary='Security and Privacy Considerations', |
| help_text=_DETAILED_HELP_TEXT, |
| subcommand_help_text={}, |
| ) |