Topic on Extension talk:AWS

Answered 1) why the current IAM permissions are not broad, 2) how to mark all uploaded S3 objects as non-public.

In my S3 bucket permissions I am trying to use the 4 "public access blocks". But when I do this MediaWiki cannot access the bucket and therefore the extension doesn't work.

Now given I am using an IAM policy I understand the "public access blocks" should not be an issue. I saw on an AWS video that an overly permissive IAM policy is considered public by AWS (I couldn't find this video as I was writing this), so I think that is what is happening here. This is surely overly-permissive. Has anyone had this issue?

The recommended IAM poilcy is:

"Action" "s3:*",

"Resource": "arn:aws:s3:::<something>/*",

"Action": [ "s3:Get*", "s3:List*" ],

"Resource": [ "arn:aws:s3:::<something>" ]

Would you agree the following permissions should work too (or am I missing needed permissions)?

"Action" [ "s3:ListObjects", "s3:GetObject", "s3:PutObject" ],

"Resource": "arn:aws:s3:::<something>/*",

"Action": [ "s3:GetObject", "s3:ListObjects" ],

"Resource": [ "arn:aws:s3:::<something>" ]

Thank you

1. It's not overly permissive. You are meant to use a separate S3 bucket for images, and if the S3 bucket contains only images, then there is no extra security added by only permitting certain operations. (I mean, with PutObject alone a malicious user can delete all files by overwriting them with zeroes, and you can't really restrict PutObject. While the very point of minimizing permissions is to reduce the possible impact of such malicious user's actions)
2. Currently the extension also does CopyObject (when image is moved and/or deleted), DeleteObject and alike. You can find all API calls that it uses by searching for client-> in the code.
3. There is no guarantee that additional API calls won't be used in the future versions. (that is, if some future change requires the use of "GetObjectVersion", then its addition won't be considered breaking anything, as the permission recommended in README is Get* - but it will break setups where only GetObject is allowed).

Understood, and thank you Edward for the prompt response.

The problem I am having is that using those permissions means that I cannot add the extra security layer which is to enforce the "4 public access blocks". While I appreciate that the risk and potential impact is low, I tend to like to increase the restrictions as much as possible.

I will add those permissions used in client-> calls only for the moment.

Question: what exactly do you mean by "4 public access blocks" in this context? What is mentioned on https://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-block-public-access.html ? These IAM permissions specify permissions of an authorized AWS user (and/or IAM instance profile), they have nothing to do with public access.

Yes, I was referring to those Public Access Policies in your link.

I used to have the Public Access Policies blocked for all S3 buckets both at an individual level and at a global level, as a security setting. Using this extension with the recommended setup makes me have to take off the global block and the block at an individual level for the bucket that is being used by the extension.

I couldn't find clear documentation to understand this issue specifically with regards to IAM, I did listen to a YouTube video where I recall someone from AWS said something along the lines of "we don't like IAM roles that have * so they might be considered public". And when I remove the public access policies block, the extension works correctly. Therefore it seems to me that there is something in my setup that is being 'considered public'.

Other IAM and Bucket Policies that I am using are not creating this issue on other buckets. For example, if I turn on the public access policies block on the bucket being used by MediaWiki, Cloudfront will still serve images from that particular S3 bucket. So Cloudfront, ironically, has no 'public permissions' despite effectively making the entire bucket publicly readable via serving all of its contents through a subdomain...

On the link you provided, there is a section that says "The meaning of 'public'", which talks about the issue of granting * related to some elements in the Bucket Policies.

Its really a pain, but to keep the desired security setting of globally denying all Public Access Policies for S3 buckets I would like to find a solution.

I was planning on doing some testing with 'more restricted permissions' to see if that solves it. Maybe restricting the 'Actions' or maybe restricting access to my VPC only for instance?

The only reason why your S3 buckets are considered "public" is because any visitor of non-private wiki can see/download the images that you have uploaded into it. This is by design (it's supposed to be public). It has nothing to do with IAM permissions that you mentioned above.

If you have a private wiki, then Extension:AWS marks all uploaded S3 objects with "private" ACL, so they are not accessible regardless of what you write in IAM.

In short, you are trying to solve a nonexistent problem. Since it has nothing to do with this extension, I can not provide further support on this matter.

[Note: _dot_ and _colon_ are used below to circumvent the ⧼abusefilter-warning-linkspam⧽]

It is not related to the images being accessible by any non-private wiki viewer. As I mention above:

"Cloudfront, ironically, has no 'public permissions' despite effectively making the entire bucket publicly readable via serving all of its contents through a subdomain..."

Why is the fact that any visitor can view/download the images not conflicting with the S3 buckets not being considered "public"? Because of the following configuration:

1. set $wgAWSBucketDomain = 'img_dot_mysite_dot_com'; as indicated in the extension's readme
2. set Cloudfront to serve the images from my bucket to 'img_dot_mysite_dot_com'
3. set a Bucket Policy in the S3 bucket to allow Cloudfront to serve the images as follows:

{

   "Version": "2008-10-17",

   "Id": "PolicyForCloudFrontPrivateContent",

   "Statement": [

       {

           "Sid": "1",

           "Effect": "Allow",

           "Principal": {

               "AWS": "arn:aws:iam::cloudfront:user/XXXXXXXXXXXXX"

           },

           "Action": "s3:GetObject",

           "Resource": "arn:aws:s3:::XXXXXXXXXXXXXXX/*"

       }

   ]

}

Here the buckets are not considered public with regards to the "4 public access blocks" for S3 but Cloudfront is granted access through the bucket policy. That read access is very well isolated from any other S3 type access, e.g. Put. Additionally it adds the extra layer of security of having the 4 public blocks enabled. This is our desired security setting.

And if I use the extension with "4 public access blocks", viewing the images in the wiki is not a problem. Any non-private wiki viewer can view the images given they are being served by AWS Cloudfront.

The problem comes when the EC2 server tries to write to the bucket. Why? it seems to me because the IAM policy is "considered public" by S3, and therefore I get an error such as the following:

Warning: doCreateInternal: S3Exception: Error executing "PutObject" on "https_colon_//s3_dot_amazonaws_dot_com/XXXXXXXXXX/thumb/XXXXXXXXXXXXXXXXX.jpg/120px-XXXXXXXXXXXXXXXXX.jpg"; AWS HTTP error: Client error: `PUT https_colon_//s3_dot_amazonaws_dot_com/XXXXXXXXXXXXXXXXX/thumb/XXXXXXXXXXXXXXXXX.jpg/120px-XXXXXXXXXXXXXXXXX.jpg` resulted in a `403 Forbidden` response: AccessDeniedAccess DeniedXXXXXX (truncated...) AccessDenied (client): Access Denied - AccessDeniedAccess Denied XXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX/seo= in /var/www/html/w/extensions/AWS/s3/AmazonS3FileBackend.php on line 1117

The IAM is not too broad.

The reason you are getting that error shouldn't be because the IAM policy is considered public by S3 but likely because the extension is trying to Put with ACL = public-read:

e.g. AmazonS3FileBackend.php:347: 'ACL' => $this->isSecure( $container ) ? 'private' : 'public-read',

Can you try the following in LocalSettings.php?

$wgFileBackends['s3']['privateWiki'] = true;

Seems to work.

Thank you!