Compare commits
1 commit
master
...
api_thrott
| Author | SHA1 | Date | |
|---|---|---|---|
|
f4a9934efd |
16 changed files with 1623 additions and 1649 deletions
20
.github/ISSUE_TEMPLATE/report-trouble.md
vendored
20
.github/ISSUE_TEMPLATE/report-trouble.md
vendored
|
|
@ -7,24 +7,6 @@ assignees: ''
|
|||
|
||||
---
|
||||
|
||||
<!--
|
||||
# BEFORE OPENING A NEW ISSUE CHECK THE EXISTING ONES AND RUN DESTREAMER WITH THE -v/--verbose flag and paste down below the output
|
||||
## PLEASE NEVER PASTE YOUR ACCESS TOKEN INTO A GITHUB ISSUE AS IT MAY CONTAIN PRIVATE INFORMATION
|
||||
|
||||
# NEVER PASTE YOUR ACCESS TOKEN INTO A GITHUB ISSUE AS IT MAY CONTAIN PRIVATE INFORMATION
|
||||
When you paste in output from destreamer, locate your access token (it looks like this: `Authorization: Bearer eyJ....<a lot more base64 encoded text>.....`) and redact it.
|
||||
|
||||
# Please fill the form below to give us some more info.
|
||||
-->
|
||||
|
||||
OS:
|
||||
|
||||
Launch command used:
|
||||
|
||||
|
||||
<details>
|
||||
<summary>Verbose log</summary>
|
||||
|
||||
```
|
||||
PASTE VERBOSE LOG HERE
|
||||
```
|
||||
</details>
|
||||
|
|
|
|||
5
.github/workflows/build.yaml
vendored
5
.github/workflows/build.yaml
vendored
|
|
@ -6,9 +6,6 @@ on:
|
|||
- 'README.md'
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
branches:
|
||||
- master
|
||||
|
||||
jobs:
|
||||
build:
|
||||
|
|
@ -17,7 +14,7 @@ jobs:
|
|||
|
||||
strategy:
|
||||
matrix:
|
||||
node-version: [10.x, 12.x, 13.x]
|
||||
node-version: [8.x, 10.x, 12.x, 13.x]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v1
|
||||
|
|
|
|||
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -9,4 +9,3 @@ node_modules
|
|||
videos
|
||||
release
|
||||
build
|
||||
yarn.lock
|
||||
5
.vscode/settings.json
vendored
5
.vscode/settings.json
vendored
|
|
@ -1,4 +1,3 @@
|
|||
{
|
||||
"eslint.enable": true,
|
||||
"typescript.tsdk": "node_modules\\typescript\\lib"
|
||||
}
|
||||
"eslint.enable": true
|
||||
}
|
||||
91
README.md
91
README.md
|
|
@ -2,24 +2,21 @@
|
|||
<img src="https://github.com/snobu/destreamer/workflows/Node%20CI/badge.svg" alt="CI build status" />
|
||||
</a>
|
||||
|
||||
**destreamer v3.0** is just around the corner. You can try out a pre-release today by cloning [this branch](https://github.com/snobu/destreamer/tree/aria2c_forRealNow).
|
||||
|
||||
|
||||
|
||||
_(Alternative artwork proposals are welcome! Submit one through an Issue.)_
|
||||
|
||||
# Saves Microsoft Stream videos for offline enjoyment
|
||||
|
||||
### v2 Release, codename _Hammer of Dawn<sup>TM</sup>_
|
||||
### v2.1 Release, codename _Hammer of Dawn<sup>TM</sup>_
|
||||
|
||||
This release would not have been possible without the code and time contributed by two distinguished developers: [@lukaarma](https://github.com/lukaarma) and [@kylon](https://github.com/kylon). Thank you!
|
||||
|
||||
### Specialized versions
|
||||
### Specialized vesions
|
||||
|
||||
- [Politecnico di Milano][polimi]: fork over at https://github.com/SamanFekri/destreamer
|
||||
- [Università di Pisa][unipi]: fork over at https://github.com/Guray00/destreamer-unipi
|
||||
- [Università della Calabria][unical]: fork over at https://github.com/peppelongo96/UnicalDown
|
||||
- [Università degli Studi di Parma][unipr]: fork over at https://github.com/vRuslan/destreamer-unipr
|
||||
|
||||
## What's new
|
||||
### v2.2
|
||||
|
|
@ -38,7 +35,7 @@ Hopefully this doesn't break the end user agreement for Microsoft Stream. Since
|
|||
|
||||
## Prereqs
|
||||
|
||||
- [**Node.js**][node]: You'll need Node.js version 8.0 or higher. A GitHub Action runs tests on all major Node versions on every commit. One caveat for Node 8, if you get a `Parse Error` with `code: HPE_HEADER_OVERFLOW` you're out of luck and you'll need to upgrade to Node 10+. PLEASE NOTE WE NO LONGER TEST BUILDS AGAINST NODE 8.x. YOU ARE ON YOUR OWN.
|
||||
- [**Node.js**][node]: You'll need Node.js version 8.0 or higher. A GitHub Action runs tests on all major Node versions on every commit. One caveat for Node 8, if you get a `Parse Error` with `code: HPE_HEADER_OVERFLOW` you're out of luck and you'll need to upgrade to Node 10+.
|
||||
- **npm**: usually comes with Node.js, type `npm` in your terminal to check for its presence
|
||||
- [**ffmpeg**][ffmpeg]: a recent version (year 2019 or above), in `$PATH` or in the same directory as this README file (project root).
|
||||
- [**git**][git]: one or more npm dependencies require git.
|
||||
|
|
@ -54,29 +51,6 @@ Note that destreamer won't run in an elevated (Administrator/root) shell. Runnin
|
|||
|
||||
**WSL** (Windows Subsystem for Linux) is not supported as it can't easily pop up a browser window. It *may* work by installing an X Window server (like [Xming][xming]) and exporting the default display to it (`export DISPLAY=:0`) before running destreamer. See [this issue for more on WSL v1 and v2][wsl].
|
||||
|
||||
## Can i plug in my own browser?
|
||||
|
||||
Yes, yes you can. This may be useful if your main browser has some authentication plugins that are required for you to logon to your Microsoft Stream tenant.
|
||||
To use your own browser for the authentication part, locate the following snippet in `src/destreamer.ts` and `src/TokenCache.ts`:
|
||||
|
||||
```typescript
|
||||
const browser: puppeteer.Browser = await puppeteer.launch({
|
||||
executablePath: getPuppeteerChromiumPath(),
|
||||
// …
|
||||
});
|
||||
```
|
||||
|
||||
Navigate to `chrome://version` in the browser you want to plug in and copy executable path from there. Use double backslash for Windows.
|
||||
|
||||
Now, change `executablePath` to reflect the path to your browser and profile (i.e. to use Microsoft Edge on Windows):
|
||||
```typescript
|
||||
executablePath: 'C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe',
|
||||
```
|
||||
|
||||
You can add `userDataDir` right after `executablePath` with the path to your browser profile (also shown in `chrome://version`) if you want that loaded as well.
|
||||
|
||||
Remember to rebuild (`npm run build`) every time you change this configuration.
|
||||
|
||||
## How to build
|
||||
|
||||
To build destreamer clone this repository, install dependencies and run the build script -
|
||||
|
|
@ -97,35 +71,31 @@ Options:
|
|||
--help Show help [boolean]
|
||||
--version Show version number [boolean]
|
||||
--username, -u The username used to log into Microsoft Stream (enabling this will fill in the email field for
|
||||
you). [string]
|
||||
--videoUrls, -i List of urls to videos or Microsoft Stream groups. [array]
|
||||
you) [string]
|
||||
--videoUrls, -i List of video urls [array]
|
||||
--inputFile, -f Path to text file containing URLs and optionally outDirs. See the README for more on outDirs.
|
||||
[string]
|
||||
--outputDirectory, -o The directory where destreamer will save your downloads. [string] [default: "videos"]
|
||||
--outputTemplate, -t The template for the title. See the README for more info.
|
||||
[string] [default: "{title} - {publishDate} {uniqueId}"]
|
||||
--keepLoginCookies, -k Let Chromium cache identity provider cookies so you can use "Remember me" during login.
|
||||
Must be used every subsequent time you launch Destreamer if you want to log in automatically.
|
||||
--outputDirectory, -o The directory where destreamer will save your downloads [string] [default: "videos"]
|
||||
--keepLoginCookies, -k Let Chromium cache identity provider cookies so you can use "Remember me" during login
|
||||
[boolean] [default: false]
|
||||
--noExperiments, -x Do not attempt to render video thumbnails in the console. [boolean] [default: false]
|
||||
--simulate, -s Disable video download and print metadata information to the console.
|
||||
--noExperiments, -x Do not attempt to render video thumbnails in the console [boolean] [default: false]
|
||||
--simulate, -s Disable video download and print metadata information to the console[boolean] [default: false]
|
||||
--verbose, -v Print additional information to the console (use this before opening an issue on GitHub)
|
||||
[boolean] [default: false]
|
||||
--verbose, -v Print additional information to the console (use this before opening an issue on GitHub).
|
||||
[boolean] [default: false]
|
||||
--closedCaptions, --cc Check if closed captions are available and let the user choose which one to download (will not
|
||||
ask if only one available). [boolean] [default: false]
|
||||
--noCleanup, --nc Do not delete the downloaded video file when an FFmpeg error occurs.[boolean] [default: false]
|
||||
--closedCaptions, --cc Check if closed captions are aviable and let the user choose which one to download (will not
|
||||
ask if only one aviable) [boolean] [default: false]
|
||||
--noCleanup, --nc Do not delete the downloaded video file when an FFmpeg error occurs [boolean] [default: false]
|
||||
--vcodec Re-encode video track. Specify FFmpeg codec (e.g. libx265) or set to "none" to disable video.
|
||||
[string] [default: "copy"]
|
||||
--acodec Re-encode audio track. Specify FFmpeg codec (e.g. libopus) or set to "none" to disable audio.
|
||||
[string] [default: "copy"]
|
||||
--format Output container format (mkv, mp4, mov, anything that FFmpeg supports).
|
||||
--format Output container format (mkv, mp4, mov, anything that FFmpeg supports)
|
||||
[string] [default: "mkv"]
|
||||
--skip Skip download if file already exists. [boolean] [default: false]
|
||||
--skip Skip download if file already exists [boolean] [default: false]
|
||||
```
|
||||
|
||||
- both --videoUrls and --inputFile also accept Microsoft Teams Groups url so if your Organization placed the videos you are interested in a group you can copy the link and Destreamer will download all the videos it can inside it! A group url looks like this https://web.microsoftstream.com/group/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
|
||||
|
||||
- Passing `--username` is optional. It's there to make logging in faster (the username field will be populated automatically on the login form).
|
||||
|
||||
- You can use an absolute path for `-o` (output directory), for example `/mnt/videos`.
|
||||
|
|
@ -170,13 +140,13 @@ These optional lines must start with white space(s).
|
|||
Usage -
|
||||
```
|
||||
https://web.microsoftstream.com/video/xxxxxxxx-aaaa-xxxx-xxxx-xxxxxxxxxxxx
|
||||
-dir="videos/lessons/week1"
|
||||
-dir=videos/lessons/week1
|
||||
https://web.microsoftstream.com/video/xxxxxxxx-aaaa-xxxx-xxxx-xxxxxxxxxxxx
|
||||
-dir="videos/lessons/week2"
|
||||
-dir=videos/lessons/week2"
|
||||
```
|
||||
|
||||
### Title template
|
||||
The `-t` option allows user to specify a custom filename for the videos.
|
||||
The `-t` option allows users to input a template string for the output file names.
|
||||
|
||||
You can use one or more of the following magic sequence which will get substituted at runtime. The magic sequence must be surrounded by curly brackets like this: `{title} {publishDate}`
|
||||
|
||||
|
|
@ -188,20 +158,8 @@ You can use one or more of the following magic sequence which will get substitut
|
|||
- `authorEmail`: E-mail of video publisher
|
||||
- `uniqueId`: An _unique-enough_ ID generated from the video metadata
|
||||
|
||||
Examples -
|
||||
Example -
|
||||
```
|
||||
Input:
|
||||
-t 'This is an example'
|
||||
|
||||
Expected filename:
|
||||
This is an example.mkv
|
||||
|
||||
Input:
|
||||
-t 'This is an example by {author}'
|
||||
|
||||
Expected filename:
|
||||
This is an example by lukaarma.mkv
|
||||
|
||||
Input:
|
||||
-t '{title} - {duration} - {publishDate} - {publishTime} - {author} - {authorEmail} - {uniqueId}'
|
||||
|
||||
|
|
@ -219,15 +177,7 @@ iTerm2 on a Mac -
|
|||
|
||||
|
||||
|
||||
By default, downloads are saved under project root `Destreamer/videos/` ( Not the system media Videos folder ), unless specified by `-o` (output directory).
|
||||
|
||||
## KNOWN BUGS
|
||||
|
||||
If you get a
|
||||
```
|
||||
[FATAL ERROR] Unknown error: exit code 4
|
||||
````
|
||||
when running destreamer, then make sure you're running a recent (post year 2019), stable version of **ffmpeg**.
|
||||
By default, downloads are saved under `videos/` unless specified by `-o` (output directory).
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
@ -246,4 +196,3 @@ Please open an [issue](https://github.com/snobu/destreamer/issues) and we'll loo
|
|||
[polimi]: https://www.polimi.it
|
||||
[unipi]: https://www.unipi.it/
|
||||
[unical]: https://www.unical.it/portale/
|
||||
[unipr]: https://www.unipr.it/
|
||||
|
|
|
|||
2727
package-lock.json
generated
2727
package-lock.json
generated
File diff suppressed because it is too large
Load diff
42
package.json
42
package.json
|
|
@ -17,34 +17,34 @@
|
|||
"author": "snobu",
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"@types/mocha": "^8.0.4",
|
||||
"@types/puppeteer": "^5.4.0",
|
||||
"@types/mocha": "^7.0.2",
|
||||
"@types/puppeteer": "^1.20.4",
|
||||
"@types/readline-sync": "^1.4.3",
|
||||
"@types/tmp": "^0.2.0",
|
||||
"@types/yargs": "^15.0.11",
|
||||
"@typescript-eslint/eslint-plugin": "^4.9.0",
|
||||
"@typescript-eslint/parser": "^4.9.0",
|
||||
"eslint": "^7.14.0",
|
||||
"mocha": "^8.2.1",
|
||||
"tmp": "^0.2.1"
|
||||
"@types/tmp": "^0.1.0",
|
||||
"@types/yargs": "^15.0.3",
|
||||
"@typescript-eslint/eslint-plugin": "^2.25.0",
|
||||
"@typescript-eslint/parser": "^2.25.0",
|
||||
"eslint": "^6.8.0",
|
||||
"mocha": "^7.1.1",
|
||||
"tmp": "^0.1.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@tedconf/fessonia": "^2.1.2",
|
||||
"@types/cli-progress": "^3.8.0",
|
||||
"@tedconf/fessonia": "^2.1.0",
|
||||
"@types/cli-progress": "^3.4.2",
|
||||
"@types/jwt-decode": "^2.2.1",
|
||||
"axios": "^0.21.2",
|
||||
"axios-retry": "^3.1.9",
|
||||
"cli-progress": "^3.8.2",
|
||||
"axios": "^0.19.2",
|
||||
"axios-retry": "^3.1.8",
|
||||
"cli-progress": "^3.7.0",
|
||||
"colors": "^1.4.0",
|
||||
"is-elevated": "^3.0.0",
|
||||
"iso8601-duration": "^1.3.0",
|
||||
"jwt-decode": "^3.1.2",
|
||||
"puppeteer": "5.5.0",
|
||||
"iso8601-duration": "^1.2.0",
|
||||
"jwt-decode": "^2.2.0",
|
||||
"puppeteer": "2.1.1",
|
||||
"readline-sync": "^1.4.10",
|
||||
"sanitize-filename": "^1.6.3",
|
||||
"terminal-image": "^1.2.1",
|
||||
"typescript": "^4.1.2",
|
||||
"winston": "^3.3.3",
|
||||
"yargs": "^16.1.1"
|
||||
"terminal-image": "^1.0.1",
|
||||
"typescript": "^3.8.3",
|
||||
"winston": "^3.3.2",
|
||||
"yargs": "^15.0.3"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -34,9 +34,6 @@ export class ApiClient {
|
|||
return true;
|
||||
}
|
||||
logger.warn(`Got HTTP code ${err?.response?.status ?? undefined}. Retrying request...`);
|
||||
logger.warn('Here is the error message: ');
|
||||
console.dir(err.response?.data);
|
||||
logger.warn('We called this URL: ' + err.response?.config.baseURL + err.response?.config.url);
|
||||
|
||||
const shouldRetry: boolean = retryCodes.includes(err?.response?.status ?? 0);
|
||||
|
||||
|
|
@ -45,11 +42,6 @@ export class ApiClient {
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Used to initialize/retrive the active ApiClient
|
||||
*
|
||||
* @param session used if initializing
|
||||
*/
|
||||
public static getInstance(session?: Session): ApiClient {
|
||||
if (!ApiClient.instance) {
|
||||
ApiClient.instance = new ApiClient(session);
|
||||
|
|
@ -58,16 +50,6 @@ export class ApiClient {
|
|||
return ApiClient.instance;
|
||||
}
|
||||
|
||||
public setSession(session: Session): void {
|
||||
if (!ApiClient.instance) {
|
||||
logger.warn("Trying to update ApiCient session when it's not initialized!");
|
||||
}
|
||||
|
||||
this.session = session;
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
/**
|
||||
* Call Microsoft Stream API. Base URL is sourced from
|
||||
* the session object and prepended automatically.
|
||||
|
|
|
|||
|
|
@ -13,24 +13,12 @@ export const argv: any = yargs.options({
|
|||
username: {
|
||||
alias: 'u',
|
||||
type: 'string',
|
||||
describe: 'The username used to log into Microsoft Stream (enabling this will fill in the email field for you).',
|
||||
demandOption: false
|
||||
},
|
||||
bmeCode: {
|
||||
alias: 'b',
|
||||
type: 'string',
|
||||
describe: 'BME címtár azonosító',
|
||||
demandOption: false
|
||||
},
|
||||
password: {
|
||||
alias: 'p',
|
||||
type: 'string',
|
||||
describe: 'BME címtár jelszó',
|
||||
describe: 'The username used to log into Microsoft Stream (enabling this will fill in the email field for you)',
|
||||
demandOption: false
|
||||
},
|
||||
videoUrls: {
|
||||
alias: 'i',
|
||||
describe: 'List of urls to videos or Microsoft Stream groups.',
|
||||
describe: 'List of video urls',
|
||||
type: 'array',
|
||||
demandOption: false
|
||||
},
|
||||
|
|
@ -42,7 +30,7 @@ export const argv: any = yargs.options({
|
|||
},
|
||||
outputDirectory: {
|
||||
alias: 'o',
|
||||
describe: 'The directory where destreamer will save your downloads.',
|
||||
describe: 'The directory where destreamer will save your downloads',
|
||||
type: 'string',
|
||||
default: 'videos',
|
||||
demandOption: false
|
||||
|
|
@ -56,43 +44,42 @@ export const argv: any = yargs.options({
|
|||
},
|
||||
keepLoginCookies: {
|
||||
alias: 'k',
|
||||
describe: 'Let Chromium cache identity provider cookies so you can use "Remember me" during login.\n' +
|
||||
'Must be used every subsequent time you launch Destreamer if you want to log in automatically.',
|
||||
describe: 'Let Chromium cache identity provider cookies so you can use "Remember me" during login',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
},
|
||||
noExperiments: {
|
||||
alias: 'x',
|
||||
describe: 'Do not attempt to render video thumbnails in the console.',
|
||||
describe: 'Do not attempt to render video thumbnails in the console',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
},
|
||||
simulate: {
|
||||
alias: 's',
|
||||
describe: 'Disable video download and print metadata information to the console.',
|
||||
describe: 'Disable video download and print metadata information to the console',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
},
|
||||
verbose: {
|
||||
alias: 'v',
|
||||
describe: 'Print additional information to the console (use this before opening an issue on GitHub).',
|
||||
describe: 'Print additional information to the console (use this before opening an issue on GitHub)',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
},
|
||||
closedCaptions: {
|
||||
alias: 'cc',
|
||||
describe: 'Check if closed captions are available and let the user choose which one to download (will not ask if only one available).',
|
||||
describe: 'Check if closed captions are aviable and let the user choose which one to download (will not ask if only one aviable)',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
},
|
||||
noCleanup: {
|
||||
alias: 'nc',
|
||||
describe: 'Do not delete the downloaded video file when an FFmpeg error occurs.',
|
||||
describe: 'Do not delete the downloaded video file when an FFmpeg error occurs',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
|
|
@ -110,13 +97,13 @@ export const argv: any = yargs.options({
|
|||
demandOption: false
|
||||
},
|
||||
format: {
|
||||
describe: 'Output container format (mkv, mp4, mov, anything that FFmpeg supports).',
|
||||
describe: 'Output container format (mkv, mp4, mov, anything that FFmpeg supports)',
|
||||
type: 'string',
|
||||
default: 'mkv',
|
||||
demandOption: false
|
||||
},
|
||||
skip: {
|
||||
describe: 'Skip download if file already exists.',
|
||||
describe: 'Skip download if file already exists',
|
||||
type: 'boolean',
|
||||
default: false,
|
||||
demandOption: false
|
||||
|
|
@ -186,8 +173,9 @@ function checkInputConflicts(videoUrls: Array<string | number> | undefined,
|
|||
|
||||
|
||||
function isOutputTemplateValid(argv: any): boolean {
|
||||
let finalTemplate: string = argv.outputTemplate;
|
||||
const elementRegEx = RegExp(/{(.*?)}/g);
|
||||
let match = elementRegEx.exec(argv.outputTemplate);
|
||||
let match = elementRegEx.exec(finalTemplate);
|
||||
|
||||
// if no template elements this fails
|
||||
if (match) {
|
||||
|
|
@ -195,25 +183,30 @@ function isOutputTemplateValid(argv: any): boolean {
|
|||
while (match) {
|
||||
if (!templateElements.includes(match[1])) {
|
||||
logger.error(
|
||||
`'${match[0]}' is not available as a template element \n` +
|
||||
`Available templates elements: '${templateElements.join("', '")}' \n`,
|
||||
`'${match[0]}' is not aviable as a template element \n` +
|
||||
`Aviable templates elements: '${templateElements.join("', '")}' \n`,
|
||||
{ fatal: true }
|
||||
);
|
||||
|
||||
process.exit(1);
|
||||
}
|
||||
match = elementRegEx.exec(argv.outputTemplate);
|
||||
match = elementRegEx.exec(finalTemplate);
|
||||
}
|
||||
}
|
||||
// bad template from user, switching to default
|
||||
else {
|
||||
logger.warn('Empty output template provided, using default one \n');
|
||||
finalTemplate = '{title} - {publishDate} {uniqueId}';
|
||||
}
|
||||
|
||||
argv.outputTemplate = sanitize(argv.outputTemplate.trim());
|
||||
argv.outputTemplate = sanitize(finalTemplate.trim());
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
export function promptUser(choices: Array<string>): number {
|
||||
const index: number = readlineSync.keyInSelect(choices, 'Which resolution/format do you prefer?');
|
||||
let index: number = readlineSync.keyInSelect(choices, 'Which resolution/format do you prefer?');
|
||||
|
||||
if (index === -1) {
|
||||
process.exit(ERROR_CODE.CANCELLED_USER_INPUT);
|
||||
|
|
|
|||
|
|
@ -3,49 +3,45 @@ export const enum ERROR_CODE {
|
|||
ELEVATED_SHELL,
|
||||
CANCELLED_USER_INPUT,
|
||||
MISSING_FFMPEG,
|
||||
OUTDATED_FFMPEG,
|
||||
UNK_FFMPEG_ERROR,
|
||||
INVALID_VIDEO_GUID,
|
||||
NO_SESSION_INFO
|
||||
}
|
||||
|
||||
|
||||
export const errors: { [key: number]: string } = {
|
||||
[ERROR_CODE.UNHANDLED_ERROR]: 'Unhandled error!\n' +
|
||||
'Timeout or fatal error, please check your downloads directory and try again',
|
||||
export const errors: {[key: number]: string} = {
|
||||
[ERROR_CODE.UNHANDLED_ERROR]: 'Unhandled error!\n' +
|
||||
'Timeout or fatal error, please check your downloads directory and try again',
|
||||
|
||||
[ERROR_CODE.ELEVATED_SHELL]: 'Destreamer cannot run in an elevated (Administrator/root) shell.\n' +
|
||||
'Please run in a regular, non-elevated window.',
|
||||
[ERROR_CODE.ELEVATED_SHELL]: 'Destreamer cannot run in an elevated (Administrator/root) shell.\n' +
|
||||
'Please run in a regular, non-elevated window.',
|
||||
|
||||
[ERROR_CODE.CANCELLED_USER_INPUT]: 'Input was cancelled by user',
|
||||
[ERROR_CODE.CANCELLED_USER_INPUT]: 'Input was cancelled by user',
|
||||
|
||||
[ERROR_CODE.MISSING_FFMPEG]: 'FFmpeg is missing!\n' +
|
||||
'Destreamer requires a fairly recent release of FFmpeg to download videos',
|
||||
[ERROR_CODE.MISSING_FFMPEG]: 'FFmpeg is missing!\n' +
|
||||
'Destreamer requires a fairly recent release of FFmpeg to download videos',
|
||||
|
||||
[ERROR_CODE.MISSING_FFMPEG]: 'The FFmpeg version currently installed is too old!\n' +
|
||||
'Destreamer requires a fairly recent release of FFmpeg to download videos',
|
||||
[ERROR_CODE.UNK_FFMPEG_ERROR]: 'Unknown FFmpeg error',
|
||||
|
||||
[ERROR_CODE.UNK_FFMPEG_ERROR]: 'Unknown FFmpeg error',
|
||||
[ERROR_CODE.INVALID_VIDEO_GUID]: 'Unable to get video GUID from URL',
|
||||
|
||||
[ERROR_CODE.INVALID_VIDEO_GUID]: 'Unable to get video GUID from URL',
|
||||
|
||||
[ERROR_CODE.NO_SESSION_INFO]: 'Could not evaluate sessionInfo on the page'
|
||||
[ERROR_CODE.NO_SESSION_INFO]: 'Could not evaluate sessionInfo on the page'
|
||||
};
|
||||
|
||||
|
||||
export const enum CLI_ERROR {
|
||||
MISSING_INPUT_ARG = 'You must specify a URLs source. \n' +
|
||||
'Valid options are -i for one or more URLs separated by space or -f for input file. \n',
|
||||
MISSING_INPUT_ARG = 'You must specify a URLs source. \n' +
|
||||
'Valid options are -i for one or more URLs separated by space or -f for input file. \n',
|
||||
|
||||
INPUT_ARG_CONFLICT = 'Too many URLs sources specified! \n' +
|
||||
'Please specify a single source, either -i or -f \n',
|
||||
INPUT_ARG_CONFLICT = 'Too many URLs sources specified! \n' +
|
||||
'Please specify a single source, either -i or -f \n',
|
||||
|
||||
INPUTFILE_WRONG_EXTENSION = 'The specified inputFile has the wrong extension \n' +
|
||||
'Please make sure to use path/to/filename.txt when useing the -f option \n',
|
||||
INPUTFILE_WRONG_EXTENSION = 'The specified inputFile has the wrong extension \n' +
|
||||
'Please make sure to use path/to/filename.txt when useing the -f option \n',
|
||||
|
||||
INPUTFILE_NOT_FOUND = 'The specified inputFile does not exists \n' +
|
||||
'Please check the filename and the path you provided \n',
|
||||
INPUTFILE_NOT_FOUND = 'The specified inputFile does not exists \n'+
|
||||
'Please check the filename and the path you provided \n',
|
||||
|
||||
INVALID_OUTDIR = 'Could not create the default/specified output directory \n' +
|
||||
'Please check directory and permissions and try again. \n'
|
||||
INVALID_OUTDIR = 'Could not create the default/specified output directory \n' +
|
||||
'Please check directory and permissions and try again. \n'
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import { AxiosResponse } from 'axios';
|
|||
export async function drawThumbnail(posterImage: string, session: Session): Promise<void> {
|
||||
const apiClient: ApiClient = ApiClient.getInstance(session);
|
||||
|
||||
const thumbnail: Buffer = await apiClient.callUrl(posterImage, 'get', null, 'arraybuffer')
|
||||
let thumbnail: Buffer = await apiClient.callUrl(posterImage, 'get', null, 'arraybuffer')
|
||||
.then((response: AxiosResponse<any> | undefined) => response?.data);
|
||||
|
||||
console.log(await terminalImage.buffer(thumbnail, { width: 70 } ));
|
||||
|
|
|
|||
|
|
@ -19,16 +19,16 @@ export class TokenCache {
|
|||
return null;
|
||||
}
|
||||
|
||||
const session: Session = JSON.parse(fs.readFileSync(this.tokenCacheFile, 'utf8'));
|
||||
let session: Session = JSON.parse(fs.readFileSync(this.tokenCacheFile, 'utf8'));
|
||||
|
||||
type Jwt = {
|
||||
[key: string]: any
|
||||
}
|
||||
const decodedJwt: Jwt = jwtDecode(session.AccessToken);
|
||||
|
||||
const now: number = Math.floor(Date.now() / 1000);
|
||||
const exp: number = decodedJwt['exp'];
|
||||
const timeLeft: number = exp - now;
|
||||
let now: number = Math.floor(Date.now() / 1000);
|
||||
let exp: number = decodedJwt['exp'];
|
||||
let timeLeft: number = exp - now;
|
||||
|
||||
if (timeLeft < 120) {
|
||||
logger.warn('Access token has expired! \n');
|
||||
|
|
@ -42,13 +42,12 @@ export class TokenCache {
|
|||
}
|
||||
|
||||
public Write(session: Session): void {
|
||||
const s: string = JSON.stringify(session, null, 4);
|
||||
fs.writeFile(this.tokenCacheFile, s, (err: any) => {
|
||||
let s: string = JSON.stringify(session, null, 4);
|
||||
fs.writeFile('.token_cache', s, (err: any) => {
|
||||
if (err) {
|
||||
return logger.error(err);
|
||||
}
|
||||
|
||||
logger.info(`Fresh access token dropped into ${this.tokenCacheFile} \n`.green);
|
||||
logger.info('Fresh access token dropped into .token_cachen \n'.green);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ export type Video = {
|
|||
outPath: string;
|
||||
totalChunks: number; // Abstraction of FFmpeg timemark
|
||||
playbackUrl: string;
|
||||
posterImageUrl: string | null;
|
||||
posterImageUrl: string;
|
||||
captionsUrl?: string
|
||||
}
|
||||
|
||||
|
|
|
|||
46
src/Utils.ts
46
src/Utils.ts
|
|
@ -11,7 +11,7 @@ import fs from 'fs';
|
|||
async function extractGuids(url: string, client: ApiClient): Promise<Array<string> | null> {
|
||||
|
||||
const videoRegex = new RegExp(/https:\/\/.*\/video\/(\w{8}-(?:\w{4}-){3}\w{12})/);
|
||||
const groupRegex = new RegExp(/https:\/\/.*\/(group|channel)\/(\w{8}-(?:\w{4}-){3}\w{12})/);
|
||||
const groupRegex = new RegExp(/https:\/\/.*\/group\/(\w{8}-(?:\w{4}-){3}\w{12})/);
|
||||
|
||||
const videoMatch: RegExpExecArray | null = videoRegex.exec(url);
|
||||
const groupMatch: RegExpExecArray | null = groupRegex.exec(url);
|
||||
|
|
@ -20,23 +20,11 @@ async function extractGuids(url: string, client: ApiClient): Promise<Array<strin
|
|||
return [videoMatch[1]];
|
||||
}
|
||||
else if (groupMatch) {
|
||||
const videoNumber: number = await client.callApi(`${groupMatch[1]}s/${groupMatch[2]}`, 'get')
|
||||
const videoNumber: number = await client.callApi(`groups/${groupMatch[1]}`, 'get')
|
||||
.then((response: AxiosResponse<any> | undefined) => response?.data.metrics.videos);
|
||||
const result: Array<string> = [];
|
||||
|
||||
// Anything above $top=100 results in 400 Bad Request
|
||||
// Use $skip to skip the first 100 and get another 100 and so on
|
||||
for (let index = 0; index <= Math.floor(videoNumber / 100); index++) {
|
||||
const partial: Array<string> = await client.callApi(
|
||||
`${groupMatch[1]}s/${groupMatch[2]}/videos?$skip=${100 * index}&` +
|
||||
'$top=100&$orderby=publishedDate asc', 'get')
|
||||
.then(
|
||||
(response: AxiosResponse<any> | undefined) =>
|
||||
response?.data.value.map((item: any) => item.id)
|
||||
);
|
||||
|
||||
result.push(...partial);
|
||||
}
|
||||
let result: Array<string> = await client.callApi(`groups/${groupMatch[1]}/videos?$top=${videoNumber}&$orderby=publishedDate asc`, 'get')
|
||||
.then((response: AxiosResponse<any> | undefined) => response?.data.value.map((item: any) => item.id));
|
||||
|
||||
return result;
|
||||
}
|
||||
|
|
@ -60,7 +48,7 @@ export async function parseCLIinput(urlList: Array<string>, defaultOutDir: strin
|
|||
session: Session): Promise<Array<Array<string>>> {
|
||||
|
||||
const apiClient: ApiClient = ApiClient.getInstance(session);
|
||||
const guidList: Array<string> = [];
|
||||
let guidList: Array<string> = [];
|
||||
|
||||
for (const url of urlList) {
|
||||
const guids: Array<string> | null = await extractGuids(url, apiClient);
|
||||
|
|
@ -97,8 +85,8 @@ export async function parseInputFile(inputFile: string, defaultOutDir: string,
|
|||
.split(/\r?\n/);
|
||||
const apiClient: ApiClient = ApiClient.getInstance(session);
|
||||
|
||||
const guidList: Array<string> = [];
|
||||
const outDirList: Array<string> = [];
|
||||
let guidList: Array<string> = [];
|
||||
let outDirList: Array<string> = [];
|
||||
// if the last line was an url set this
|
||||
let foundUrl = false;
|
||||
|
||||
|
|
@ -113,23 +101,23 @@ export async function parseInputFile(inputFile: string, defaultOutDir: string,
|
|||
// parse if line is option
|
||||
else if (line.includes('-dir')) {
|
||||
if (foundUrl) {
|
||||
const outDir: string | null = parseOption('-dir', line);
|
||||
let outDir: string | null = parseOption('-dir', line);
|
||||
|
||||
if (outDir && checkOutDir(outDir)) {
|
||||
outDirList.push(...Array(guidList.length - outDirList.length)
|
||||
.fill(outDir));
|
||||
.fill(outDir));
|
||||
}
|
||||
else {
|
||||
outDirList.push(...Array(guidList.length - outDirList.length)
|
||||
.fill(defaultOutDir));
|
||||
.fill(defaultOutDir));
|
||||
}
|
||||
|
||||
foundUrl = false;
|
||||
continue;
|
||||
}
|
||||
else {
|
||||
logger.warn(`Found options without preceding url at line ${i + 1}, skipping..`);
|
||||
continue;
|
||||
logger.warn(`Found options without preceding url at line ${i + 1}, skipping..`);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -167,7 +155,7 @@ export async function parseInputFile(inputFile: string, defaultOutDir: string,
|
|||
function parseOption(optionSyntax: string, item: string): string | null {
|
||||
const match: RegExpMatchArray | null = item.match(
|
||||
RegExp(`^\\s*${optionSyntax}\\s?=\\s?['"](.*)['"]`)
|
||||
);
|
||||
);
|
||||
|
||||
return match ? match[1] : null;
|
||||
}
|
||||
|
|
@ -180,7 +168,7 @@ export function checkOutDir(directory: string): boolean {
|
|||
logger.info('\nCreated directory: '.yellow + directory);
|
||||
}
|
||||
catch (e) {
|
||||
logger.warn('Cannot create directory: ' + directory +
|
||||
logger.warn('Cannot create directory: '+ directory +
|
||||
'\nFalling back to default directory..');
|
||||
|
||||
return false;
|
||||
|
|
@ -193,13 +181,7 @@ export function checkOutDir(directory: string): boolean {
|
|||
|
||||
export function checkRequirements(): void {
|
||||
try {
|
||||
const copyrightYearRe = new RegExp(/\d{4}-(\d{4})/);
|
||||
const ffmpegVer: string = execSync('ffmpeg -version').toString().split('\n')[0];
|
||||
|
||||
if (parseInt(copyrightYearRe.exec(ffmpegVer)?.[1] ?? '0') <= 2019) {
|
||||
process.exit(ERROR_CODE.OUTDATED_FFMPEG);
|
||||
}
|
||||
|
||||
logger.verbose(`Using ${ffmpegVer}\n`);
|
||||
}
|
||||
catch (e) {
|
||||
|
|
|
|||
|
|
@ -24,14 +24,14 @@ function publishedTimeToString(date: string): string {
|
|||
const minutes: string = dateJs.getMinutes().toString();
|
||||
const seconds: string = dateJs.getSeconds().toString();
|
||||
|
||||
return `${hours}.${minutes}.${seconds}`;
|
||||
return `${hours}:${minutes}:${seconds}`;
|
||||
}
|
||||
|
||||
|
||||
function isoDurationToString(time: string): string {
|
||||
const duration: Duration = parseDuration(time);
|
||||
|
||||
return `${duration.hours ?? '00'}.${duration.minutes ?? '00'}.${duration.seconds?.toFixed(0) ?? '00'}`;
|
||||
return `${duration.hours ?? '00'}:${duration.minutes ?? '00'}:${duration.seconds?.toFixed(0) ?? '00'}`;
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -45,8 +45,8 @@ function durationToTotalChunks(duration: string): number {
|
|||
}
|
||||
|
||||
|
||||
export async function getVideoInfo(videoGuids: Array<string>, session: Session, subtitles?: boolean): Promise<Array<Video>> {
|
||||
const metadata: Array<Video> = [];
|
||||
export async function getVideoInfo(videoGuid: string, session: Session, subtitles?: boolean): Promise<Video> {
|
||||
// template elements
|
||||
let title: string;
|
||||
let duration: string;
|
||||
let publishDate: string;
|
||||
|
|
@ -54,113 +54,101 @@ export async function getVideoInfo(videoGuids: Array<string>, session: Session,
|
|||
let author: string;
|
||||
let authorEmail: string;
|
||||
let uniqueId: string;
|
||||
// final video path (here for consistency with typedef)
|
||||
const outPath = '';
|
||||
// ffmpeg magic (abstraction of FFmpeg timemark)
|
||||
let totalChunks: number;
|
||||
// various sources
|
||||
let playbackUrl: string;
|
||||
let posterImageUrl: string;
|
||||
let captionsUrl: string | undefined;
|
||||
|
||||
const apiClient: ApiClient = ApiClient.getInstance(session);
|
||||
|
||||
/* TODO: change this to a single guid at a time to ease our footprint on the
|
||||
MSS servers or we get throttled after 10 sequential reqs */
|
||||
for (const guid of videoGuids) {
|
||||
const response: AxiosResponse<any> | undefined =
|
||||
await apiClient.callApi('videos/' + guid + '?$expand=creator', 'get');
|
||||
let response: AxiosResponse<any> | undefined =
|
||||
await apiClient.callApi('videos/' + videoGuid + '?$expand=creator', 'get');
|
||||
|
||||
title = sanitizeWindowsName(response?.data['name']);
|
||||
title = sanitizeWindowsName(response?.data['name']);
|
||||
|
||||
duration = isoDurationToString(response?.data.media['duration']);
|
||||
duration = isoDurationToString(response?.data.media['duration']);
|
||||
|
||||
publishDate = publishedDateToString(response?.data['publishedDate']);
|
||||
publishDate = publishedDateToString(response?.data['publishedDate']);
|
||||
|
||||
publishTime = publishedTimeToString(response?.data['publishedDate']);
|
||||
publishTime = publishedTimeToString(response?.data['publishedDate']);
|
||||
|
||||
author = response?.data['creator'].name;
|
||||
author = response?.data['creator'].name;
|
||||
|
||||
authorEmail = response?.data['creator'].mail;
|
||||
authorEmail = response?.data['creator'].mail;
|
||||
|
||||
uniqueId = '#' + guid.split('-')[0];
|
||||
uniqueId = '#' + videoGuid.split('-')[0];
|
||||
|
||||
totalChunks = durationToTotalChunks(response?.data.media['duration']);
|
||||
totalChunks = durationToTotalChunks(response?.data.media['duration']);
|
||||
|
||||
playbackUrl = response?.data['playbackUrls']
|
||||
.filter((item: { [x: string]: string; }) =>
|
||||
item['mimeType'] == 'application/vnd.apple.mpegurl')
|
||||
.map((item: { [x: string]: string }) => {
|
||||
return item['playbackUrl'];
|
||||
})[0];
|
||||
playbackUrl = response?.data['playbackUrls']
|
||||
.filter((item: { [x: string]: string; }) =>
|
||||
item['mimeType'] == 'application/vnd.apple.mpegurl')
|
||||
.map((item: { [x: string]: string }) => {
|
||||
return item['playbackUrl'];
|
||||
})[0];
|
||||
|
||||
posterImageUrl = response?.data['posterImage']['medium']['url'];
|
||||
posterImageUrl = response?.data['posterImage']['medium']['url'];
|
||||
|
||||
if (subtitles) {
|
||||
const captions: AxiosResponse<any> | undefined = await apiClient.callApi(`videos/${guid}/texttracks`, 'get');
|
||||
if (subtitles) {
|
||||
let captions: AxiosResponse<any> | undefined = await apiClient.callApi(`videos/${videoGuid}/texttracks`, 'get');
|
||||
|
||||
if (!captions?.data.value.length) {
|
||||
captionsUrl = undefined;
|
||||
}
|
||||
else if (captions?.data.value.length === 1) {
|
||||
logger.info(`Found subtitles for ${title}. \n`);
|
||||
captionsUrl = captions?.data.value.pop().url;
|
||||
}
|
||||
else {
|
||||
const index: number = promptUser(captions.data.value.map((item: { language: string; autoGenerated: string; }) => {
|
||||
return `[${item.language}] autogenerated: ${item.autoGenerated}`;
|
||||
}));
|
||||
captionsUrl = captions.data.value[index].url;
|
||||
}
|
||||
if (!captions?.data.value.length) {
|
||||
captionsUrl = undefined;
|
||||
}
|
||||
else if (captions?.data.value.length === 1) {
|
||||
logger.info(`Found subtitles for ${title}. \n`);
|
||||
captionsUrl = captions?.data.value.pop().url;
|
||||
}
|
||||
else {
|
||||
const index: number = promptUser(captions.data.value.map((item: { language: string; autoGenerated: string; }) => {
|
||||
return `[${item.language}] autogenerated: ${item.autoGenerated}`;
|
||||
}));
|
||||
captionsUrl = captions.data.value[index].url;
|
||||
}
|
||||
|
||||
metadata.push({
|
||||
title: title,
|
||||
duration: duration,
|
||||
publishDate: publishDate,
|
||||
publishTime: publishTime,
|
||||
author: author,
|
||||
authorEmail: authorEmail,
|
||||
uniqueId: uniqueId,
|
||||
outPath: outPath,
|
||||
totalChunks: totalChunks, // Abstraction of FFmpeg timemark
|
||||
playbackUrl: playbackUrl,
|
||||
posterImageUrl: posterImageUrl,
|
||||
captionsUrl: captionsUrl
|
||||
});
|
||||
}
|
||||
|
||||
return metadata;
|
||||
return {
|
||||
title: title,
|
||||
duration: duration,
|
||||
publishDate: publishDate,
|
||||
publishTime: publishTime,
|
||||
author: author,
|
||||
authorEmail: authorEmail,
|
||||
uniqueId: uniqueId,
|
||||
outPath: outPath,
|
||||
totalChunks: totalChunks,
|
||||
playbackUrl: playbackUrl,
|
||||
posterImageUrl: posterImageUrl,
|
||||
captionsUrl: captionsUrl
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
export function createUniquePath(videos: Array<Video>, outDirs: Array<string>, template: string, format: string, skip?: boolean): Array<Video> {
|
||||
export function createUniquePath(video: Video, outDir: string, template: string, format: string, skip?: boolean): Video {
|
||||
|
||||
videos.forEach((video: Video, index: number) => {
|
||||
let title: string = template;
|
||||
let finalTitle: string;
|
||||
const elementRegEx = RegExp(/{(.*?)}/g);
|
||||
let match = elementRegEx.exec(template);
|
||||
let title: string = template;
|
||||
let finalTitle: string;
|
||||
const elementRegEx = RegExp(/{(.*?)}/g);
|
||||
let match = elementRegEx.exec(template);
|
||||
|
||||
while (match) {
|
||||
const value = video[match[1] as keyof Video] as string;
|
||||
title = title.replace(match[0], value);
|
||||
match = elementRegEx.exec(template);
|
||||
}
|
||||
while (match) {
|
||||
let value = video[match[1] as keyof Video] as string;
|
||||
title = title.replace(match[0], value);
|
||||
match = elementRegEx.exec(template);
|
||||
}
|
||||
|
||||
let i = 0;
|
||||
finalTitle = title;
|
||||
let i = 0;
|
||||
finalTitle = title;
|
||||
|
||||
while (!skip && fs.existsSync(path.join(outDirs[index], finalTitle + '.' + format))) {
|
||||
finalTitle = `${title}.${++i}`;
|
||||
}
|
||||
while (!skip && fs.existsSync(path.join(outDir, finalTitle + '.' + format))) {
|
||||
finalTitle = `${title}.${++i}`;
|
||||
}
|
||||
|
||||
const finalFileName = `${finalTitle}.${format}`;
|
||||
const cleanFileName = sanitizeWindowsName(finalFileName, { replacement: '_' });
|
||||
if (finalFileName !== cleanFileName) {
|
||||
logger.warn(`Not a valid Windows file name: "${finalFileName}".\nReplacing invalid characters with underscores to preserve cross-platform consistency.`);
|
||||
}
|
||||
video.outPath = path.join(outDir, finalTitle + '.' + format);
|
||||
|
||||
video.outPath = path.join(outDirs[index], finalFileName);
|
||||
|
||||
});
|
||||
|
||||
return videos;
|
||||
return video;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -13,7 +13,6 @@ import cliProgress from 'cli-progress';
|
|||
import fs from 'fs';
|
||||
import isElevated from 'is-elevated';
|
||||
import puppeteer from 'puppeteer';
|
||||
import { ApiClient } from './ApiClient';
|
||||
|
||||
|
||||
const { FFmpegCommand, FFmpegInput, FFmpegOutput } = require('@tedconf/fessonia')();
|
||||
|
|
@ -37,9 +36,6 @@ async function init(): Promise<void> {
|
|||
if (argv.username) {
|
||||
logger.info(`Username: ${argv.username}`);
|
||||
}
|
||||
if(argv.bmeCode){
|
||||
logger.info(`BME Címtár azonosító: ${argv.bmeCode}`);
|
||||
}
|
||||
|
||||
if (argv.simulate) {
|
||||
logger.warn('Simulate mode, there will be no video downloaded. \n');
|
||||
|
|
@ -47,7 +43,7 @@ async function init(): Promise<void> {
|
|||
}
|
||||
|
||||
|
||||
async function DoInteractiveLogin(url: string, username?: string,bmeCode?: string,password?: string): Promise<Session> {
|
||||
async function DoInteractiveLogin(url: string, username?: string): Promise<Session> {
|
||||
|
||||
logger.info('Launching headless Chrome to perform the OpenID Connect dance...');
|
||||
|
||||
|
|
@ -71,16 +67,6 @@ async function DoInteractiveLogin(url: string, username?: string,bmeCode?: strin
|
|||
await page.waitForSelector('input[type="email"]', {timeout: 3000});
|
||||
await page.keyboard.type(username);
|
||||
await page.click('input[type="submit"]');
|
||||
if(bmeCode){
|
||||
await page.waitForSelector('input[type="text"]', {timeout: 3000});
|
||||
await page.type('input[type="text"]', bmeCode);
|
||||
if(password){
|
||||
await page.type('input[type="password"]', password);
|
||||
await page.click('input[type="submit"]');
|
||||
await page.waitForSelector('#idSIButton9');
|
||||
await page.click("#idSIButton9");
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
/* If a username was not provided we let the user take actions that
|
||||
|
|
@ -133,16 +119,17 @@ async function DoInteractiveLogin(url: string, username?: string,bmeCode?: strin
|
|||
}
|
||||
|
||||
|
||||
async function downloadVideo(videoGUIDs: Array<string>, outputDirectories: Array<string>, session: Session): Promise<void> {
|
||||
async function downloadVideo(videoGuidArray: Array<string>, outputDirectoryArray: Array<string>, session: Session): Promise<void> {
|
||||
|
||||
logger.info('Fetching videos info... \n');
|
||||
const videos: Array<Video> = createUniquePath (
|
||||
await getVideoInfo(videoGUIDs, session, argv.closedCaptions),
|
||||
outputDirectories, argv.outputTemplate, argv.format, argv.skip
|
||||
for (const [index, videoGuid] of videoGuidArray.entries()) {
|
||||
logger.info(`Fetching video's #${index} info... \n`);
|
||||
|
||||
const video: Video = createUniquePath (
|
||||
await getVideoInfo(videoGuid, session, argv.closedCaptions),
|
||||
outputDirectoryArray[index], argv.outputTemplate, argv.format, argv.skip
|
||||
);
|
||||
|
||||
if (argv.simulate) {
|
||||
videos.forEach((video: Video) => {
|
||||
if (argv.simulate) {
|
||||
logger.info(
|
||||
'\nTitle: '.green + video.title +
|
||||
'\nOutPath: '.green + video.outPath +
|
||||
|
|
@ -150,22 +137,19 @@ async function downloadVideo(videoGUIDs: Array<string>, outputDirectories: Array
|
|||
'\nPlayback URL: '.green + video.playbackUrl +
|
||||
((video.captionsUrl) ? ('\nCC URL: '.green + video.captionsUrl) : '')
|
||||
);
|
||||
});
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
for (const [index, video] of videos.entries()) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (argv.skip && fs.existsSync(video.outPath)) {
|
||||
logger.info(`File already exists, skipping: ${video.outPath} \n`);
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (argv.keepLoginCookies && index !== 0) {
|
||||
logger.info('Trying to refresh token...');
|
||||
session = await refreshSession('https://web.microsoftstream.com/video/' + videoGUIDs[index]);
|
||||
ApiClient.getInstance().setSession(session);
|
||||
session = await refreshSession('https://web.microsoftstream.com/video/' + videoGuidArray[index]);
|
||||
}
|
||||
|
||||
const pbar: cliProgress.SingleBar = new cliProgress.SingleBar({
|
||||
|
|
@ -198,9 +182,7 @@ async function downloadVideo(videoGUIDs: Array<string>, outputDirectories: Array
|
|||
const headers: string = 'Authorization: Bearer ' + session.AccessToken;
|
||||
|
||||
if (!argv.noExperiments) {
|
||||
if (video.posterImageUrl) {
|
||||
await drawThumbnail(video.posterImageUrl, session);
|
||||
}
|
||||
await drawThumbnail(video.posterImageUrl, session);
|
||||
}
|
||||
|
||||
const ffmpegInpt: any = new FFmpegInput(video.playbackUrl, new Map([
|
||||
|
|
@ -285,8 +267,7 @@ async function main(): Promise<void> {
|
|||
await init(); // must be first
|
||||
|
||||
let session: Session;
|
||||
// eslint-disable-next-line prefer-const
|
||||
session = tokenCache.Read() ?? await DoInteractiveLogin('https://web.microsoftstream.com/', argv.username,argv.bmeCode,argv.password);
|
||||
session = tokenCache.Read() ?? await DoInteractiveLogin('https://web.microsoftstream.com/', argv.username);
|
||||
|
||||
logger.verbose('Session and API info \n' +
|
||||
'\t API Gateway URL: '.cyan + session.ApiGatewayUri + '\n' +
|
||||
|
|
|
|||
Loading…
Reference in a new issue