Anyone has done it using Cloudflare R2?
It's almost similar to those other cloud, but it seems to be simpler.
Extension talk:AWS
Hello, I'm working with a private wiki and a private S3 bucket. To display the S3 images in the wiki, does the bucket need read access? Or can I use the extension while keeping everything private? Thanks!
It doesn't need read access. Private wikis serve images via /img_auth.php, not directly.
I'm getting the above error. can anyone assist? Here is my LocalSettings (scrubbed for privacy)
$wgFileBackends['s3'];
wfLoadExtension( 'AWS' );
// Configure AWS credentials.
// THIS IS NOT NEEDED if your EC2 instance has an IAM instance profile.
$wgAWSCredentials = [
'key' => xx,
'secret' => 'xxx',
'token' => false
];
$wgAWSRegion = 'us-east-1'; # Northern Virginia
// Replace <something> with the name of your S3 bucket, e.g. wonderfulbali234.
$wgAWSBucketName = "xxx";
and this is the policy we have
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::<bucketname>*"
},
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": "arn:aws:s3:::<bucketname>"
}
]
I removed the credentials from our LocalSettings file and tried to just use our IAM and now I'm getting this error
[746bd41fcda522fdafb85fb8] /wiki/Special:Upload Aws\Exception\CredentialsException: Error retrieving credentials from the instance profile metadata service. (cURL error 28: Connection timed out after 1001 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://169.254.169.254/latest/meta-data/iam/security-credentials/)
> Connection timed out after 1001 milliseconds
This looks like a firewall is blocking a connection, or some URL is set incorrectly
(Also posted in GitHub issues for this repo)
I recently started getting these errors and I am struggling to figure out why.
Nothing has changed in my AWS configuration. The IAM configuration is still good and all of the bucket settings have not changed.
I am on PHP 7.4, MediaWiki 1.35, Extension:AWS 0.11.1. This hasn't really changed either.
I did recently update my composer dependencies. Per the MediaWiki documentation I removed my composer.lock file and ran composer install
Files are still being read from the bucket correctly.
Does anyone have troubleshooting suggests or know what the issue is?
I verified that the AWS credentials I am using are still working correctly. I also tried using the latest code from the extension's repo.
To be clear, this was working just fine a few weeks ago and the only thing that has changed since then was that I updated the composer dependencies and I enabled the VisualEditor functionality.
Here is the error I am seeing in the debug logs (some information obfuscated):
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF (truncated...)
AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF*******</RequestId><HostId>u6uU*************************************************</HostId></Error>
[error] [de39d4fe79d16409eda7a6cf] /wiki/Special:Upload ErrorException from line 1104 of /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php: PHP Warning: doCreateInternal: S3Exception: Error executing "PutObject" on "******/Shopify_Photoshop_Actions.atn.zip"; AWS HTTP error: Client error: `PUT *******/Shopify_Photoshop_Actions.atn.zip` resulted in a `403 Forbidden` response:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF (truncated...)
AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>NM94VF*******</RequestId><HostId>u6uU*************************************************</HostId></Error>
#0 [internal function]: MWExceptionHandler::handleError()
#1 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(1104): trigger_error()
#2 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(1031): AmazonS3FileBackend->logException()
#3 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(347): AmazonS3FileBackend->runWithExceptionHandling()
#4 /var/www/html/extensions/AWS/s3/AmazonS3FileBackend.php(369): AmazonS3FileBackend->doCreateInternal()
#5 /var/www/html/includes/libs/filebackend/FileBackendStore.php(187): AmazonS3FileBackend->doStoreInternal()
#6 /var/www/html/includes/libs/filebackend/fileop/StoreFileOp.php(74): FileBackendStore->storeInternal()
#7 /var/www/html/includes/libs/filebackend/fileop/FileOp.php(301): StoreFileOp->doAttempt()
#8 /var/www/html/includes/libs/filebackend/FileOpBatch.php(176): FileOp->attempt()
#9 /var/www/html/includes/libs/filebackend/FileOpBatch.php(132): FileOpBatch::runParallelBatches()
#10 /var/www/html/includes/libs/filebackend/FileBackendStore.php(1308): FileOpBatch::attempt()
...And here are the versions Composer is using for the extension's dependencies:
- Locking aws/aws-sdk-php (3.209.17)
- Locking composer/installers (v1.12.0)The issue is that my Wiki and bucket are private and I did not have $wgFileBackends['s3']['privateWiki'] = true; in my local settings. I am still not sure yet how this was working before and then stopped but... it is working now.
Thanks to Edward for helping me figure it out on GitHub.
What kind of wiki is this good for?
Wikis in such environments
Been looking at this and it seems the JSON for the IAM role isn't correct anymore. Maybe amazon changed their grammar policy since the original entry this is what I have got to:
{
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::<something>/*"
},
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": "arn:aws:s3:::<something>"
}
]
}
Nothing changed. The example in the article was always supposed to be inserted into the Statement array. This is not a "replace IAM inline policy with this" example, because IAM inline policy may already exist (and contain other rules that shouldn't be overwritten).
I think the OP's sentiment is valid. Not sure why the documentation doesn't include this. It would make the setup less confusing to deal with for those who are new to S3/IAM.
Upon inspection, it did, but as a citation. Since this is very important, I've taken it out of a footnote.
Is there an example file that will work for someone who created a brand-new bucket just for this?
I'm asking about this due to hearing about increased security issues regarding AWS, and I want to keep things locked down while still enabling regular use of Mediawiki.
[Edited to make things more clear and reduce confusion]
@DiscordiaChaos The above JSON, in its exact form (apart from the ARN needing to be filled in), should be safe.
Doesn't seem to work very well with the pdf thumbnail pages generated by default by mediawiki, this is probably just a function of the realities of generating of a pdf for a file on an s3 store just don't make a lot of sense and probably should be processed as a batch job than done on the fly like configured by default.
This seeks to clarify this diff since I fail to understand your edit comment. Note that MediaWiki master is now at 1.37+. Your change means that the extension caters for MW 1.35, 1.36 and 1.37? Not sure if this is meant but it will be cool if this is correct I believe.
The extension itself uses "master" compatibility policy, meaning that it maintains backward compatibility with MediaWiki 1.35. Its repository has REL1_34 branch that supports 1.27-1.34.
Cool, thanks for clarifying.
Does this extension also allow use of API-compatible backends like MinIO, Linode Object Storage, etc.?
It does (see the README file for examples).
Hi, I would like to have mediawiki store uploaded datasets, csv files, and zip files (and in general any file) to AWS. Can I do this with this extension, or is it only for images (and videos?) Thanks
If you can upload it to your wiki, this extension will put that file on s3. Does that help?
Thank you @T0lk, so this extension puts ALL uploads, regardless of their file type on s3?
Yes. This works for all uploads (from Special:Upload)
Thank you for the reply @Ciencia Al Poder. Can I somehow also integrate it with the upload through visual editor? (i.e. when you edit a page with visual editor, and you press Insert > Media > Upload ?
AFAIK, it affects *all uploads* to the local wiki, no matter how they're uploaded (special:upload was an example), since this is the common repository for the wiki, and there's no way to choose between file repositories on upload
Thank you for the information , much appreciated.