⭐️ Basic Link Navigation ⭐️
Server →☁️ AliCloud event address
See sample →🐟 Groping small website address
Learning Code →💻 Source code repository address
I. Preface
To this article we finally set up the environment, also made a few hot search widgets, in order to make it easy for us to show the results or easy to touch the fish themselves, we need to deploy this small site on the cloud. The overall process is not complicated, but there are many small details. If a certain detail is not handled properly, it may lead to deployment failure, so this is a process of continuous trial and adjustment. The basic process consists of modifying the configuration, packaging, uploading, running and debugging, and then iterating until it works.
Second, front-end resource packaging
1. Modification of the configuration
locatefile, change the baseURL to the IP address of your own server, the port is recommended to use 80, because the default port of the http protocol is 80, the default port of the https protocol is 443, this should be known, as follows:
//
import axios from "axios";
// establishaxiosExample and configure the baseURL
const apiClient = ({
baseURL: "http://ip:80/api",
headers: {
"Content-Type": "application/json"
},
});
export default {
// seal insideGetconnector
get(fetchUrl) {
return (fetchUrl);
}
};
locatefile, replace the following configuration, mainly used to optimize the packaged file, remove some unnecessary files, as follows:
const { defineConfig } = require('@vue/cli-service')
= defineConfig({
transpileDependencies: true, // Remove some .map files generated in the js directory after Vue is packaged.
// Remove some of the .map files generated in the js directory after Vue is packaged to speed up production builds.
productionSourceMap: false, // Remove some .map files generated in the js directory after Vue is packaged.
// Remove the 8-bit hash from the names of the .css and .js files in the Vue package, which is related to caching; usually true will do.
filenameHashing:false
})
2. Execution of packing orders
Open the command line and enter the pack command as follows:
npm run build
At the end of packaging, there will be a dist folder with html, js, css and other files as follows:
3. Providing a resource access interface
existsummo-sbmy-startCreate a static file in the resources directory of this module and copy the contents of the dist into it (ps: not including the dist directory itself) as follows:
Add the thymeleaf configuration in as follows:
=classpath:/static/
=.html
=HTML
end up in the middle ofsummo-sbmy-webThis module is created as follows:
package ;
import ;
import ;
@Controller
public class IndexController {
@GetMapping("/")
public String index(){
return "index";
}
}
Once these are done, launch the application and access thehttp://localhost/, should be accessible if there are no problems.
If you find that the icons don't load, there's probably a packaging configuration that isn't working properly, set the<include>**/*.svg</include>This line is placed in thesummo-sbmy-startThis module's<build>tagged as follows:
<resources>
<resource>
<!-- Specifies that the configuration file is in theresourcecatalogs -->
<directory>src/main/resources</directory>
<includes>
<include></include>
<include></include>
<include>**/*.html</include>
<include>**/*.js</include>
<include>**/*.css</include>
<include>**/*.svg</include>
</includes>
<filtering>true</filtering>
</resource>
<resource>
<!-- Specifies that the configuration file is in theresourcecatalogs -->
<directory>src/main/resources</directory>
<includes>
<include>**/*.woff</include>
<include>**/*.ttf</include>
<include>**/*.xdb</include>
<include>**/*.jks</include>
<include>**/*.svg</include>
</includes>
<filtering>false</filtering>
</resource>
</resources>
Once the configuration and interface are done, start the app and revisit it, you should see the icon at this point.
III. Back-end application packaging
Before packing and uploading let's make some small preparations, open our idea tool, find the plugin management, usually File -> Settings -> Plugins, and search forAlibaba Cloud Toolkit, download and install it as follows:
After downloading and installing, there is an Alibaba Cloud View, where you can directly connect to the Aliyun ECS server we bought as follows:
This plugin allows you to upload files and also open the terminal directly so that you don't have to download special software to connect to ECS, which is very convenient.
1. xxl-job application deployment
(1) Application Packaging
Find the Maven plugin and execute the package command as follows:
Pack it out as follows:
(2) Application Upload
Using the AliCloud plugin you just downloaded, you can directly upload the packaged jar packages to the specified directory, and I set up the/home/admin/xxl-job, as follows:
After uploading, open the terminal, you can start it and see if there are any problems as follows:
(3) Execution of scripts
After we use the SpringBoot framework to develop a project, the project needs to be typed into a jar package to the server for production to run. Generally is to execute java -jar & command to run, but this is problematic.
For example, if you need to add parameters at startup, such as -Dxxx=xxx, the command will be long, unreadable and forgettable. Therefore, it is better to use shell scripts to maintain the configuration and commands.
Create a new shell script called, as follows:
/bin/bash /bin/bash
# Replace this with your own executable program, no other code needs to be changed.
APP_NAME=
#Usage instructions to prompt for parameters
usage() {
echo "Usage: sh script name.sh [start|stop|restart|status]"
exit 1
}
# Check if the program is running
is_exist(){
pid=`ps -ef|grep $APP_NAME|grep -v grep|awk '{print 2}' `
# Return 1 if not present, 0 if present
if [ -z "${pid}" ]; then
return 1
if [ -z "${pid}" ]; then return 1
return 0
fi
}
#The start method
start(){
is_exist
if [ $? -eq "0" ]; then
echo "${APP_NAME} is already running. pid=${pid} ."
then echo "${APP_NAME} is already running.
nohup java -jar /home/admin/$APP_NAME > /dev/null 2>&1 &
echo "${APP_NAME} start success"
}
}
# Stop methods
stop(){
is_exist
if [ $? -eq "0" ]; then
kill -9 $pid
else
echo "${APP_NAME} is not running"
fi
}
# Output running status
status(){
is_exist
if [ $? -eq "0" ]; then
echo "${APP_NAME} is running. Pid is ${pid}"
then echo "${APP_NAME} is running.
echo "${APP_NAME} is NOT running."
fi
}
#Restart
restart(){
restart(){
start
}
# Choose to execute the corresponding method based on the input parameters, or execute the usage instructions without inputting them
case "$1" in
"start")
start
;;.
"stop")
stop
start ;; "stop") stop
"status")
status
status")
"restart")
restart
;;
*restart ;; "restart")
usage
;;.
esac
Use it as follows:
- Give document authorization:
chmod 744 - Start the service and execute it in the current directory:
./ start - Shut down the service and execute it in the current directory:
./ stop - Restart the service and execute it in the current directory:
./ restart - Check the status of the service, executed in the current directory:
./ status
2. summo-sbmy application deployment
Packing, uploading, and launching are the same steps as the xxl-job app above, so I won't go into that.
Here's a little something else:How do I specify a profile when packaging?
It's that for development we usually have two configuration files (or more): the daily one; the online one. These two files are configured the same, but use different resources, two sets of MySQL, Redis, etc. for development and online, like this, below:
Some of you may be wondering why we need this file when we have configuration files for our daily and online environments.
Here we can think of it as a configuration file container, which can add the contents of other configuration files to it. Another reason is because the SpringBoot project starts only recognizes the file and nothing else.
(1) Differentiation of environment configuration
First we add the environment configuration under this module as follows:
<!-- Add a variable to the environment，variable namedenvironment -->
<profiles>
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<environment>dev</environment>
</properties>
</profile>
<profile>
<id>publish</id>
<properties>
<environment>publish</environment>
</properties>
</profile>
</profiles>
<build>
<finalName>summo-sbmy</finalName>
<resources>
<resource>
<!-- Specifies that the configuration file is in theresourcecatalogs -->
<directory>src/main/resources</directory>
<includes>
<include></include>
<include>application-${environment}.properties</include>
</includes>
<filtering>true</filtering>
</resource>
... Other configurations
</resources>
</build>
Refresh the Maven plugin and these two configurations will be scanned as follows:
(2) Use of designated environments
Add to the file=@environment@Configurations that will be unchanged can be put in as well, as follows:
#Activate the environment by configuring
=@environment@
# Project name
=summo-sbmy
# Server port
=80
# Resource scanning configuration
=classpath:/static/
=.html
=HTML
# Mybatis auto-mapping
-mapping-behavior=full
# Mybatis automatically maps underscore to camel case
-underscore-to-camel-case=true
# Path to XML mapping file for MyBatis-Plus Mapper
-locations=classpath*:/mybatis/mapper/*.xml
(3) Packing command
- Everyday Environment Packaging Directive
mvn clean package -=true -P=daily
- Formal Environment Packaging Directive
mvn clean package -=true -P=publish
IV. To summarize
The above says to specify the configuration file during packaging, but if you are too much trouble, you can also just use a copy of thefile and save yourself the trouble. I have two sets of databases for the development environment and the online environment, so I use different configuration files.
After the package is typed and uploaded to the cloud server this step is not difficult, the startup method is recommended using my recommendedScript, this shouldn't be too hard either. The Aliyun ECS security group rules seem to be fully liberalized for ports 80 and 443 by default, so as long as you don't have any problems starting it up, you'll be able to access reach your little website by typing in your machine's public IP into your browser.
Because there are a lot of details about the deployment, you may have problems of one kind or another, so feel free to leave a message in the comments section if you have questions.
Extra: Baidu Hotlist Crawler
1. Evaluation of the reptile program
The Baidu Hotlist is this, and the interface is this:/hottopic/browse/topicList?res_type=1
This is another web version of the hot list, the web version of the data parsing requires the user to jsoup, the use of this tool has been said earlier, I put the code directly.
2. Web page parsing code
package ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
import .slf4j.Slf4j;
import .;
import .;
import ;
import ;
import ;
import ;
import ;
import static .CACHE_MAP;
import static ;
/**
* @author summo
* @version , 1.0.0
* @description Internet search engineJavacrawler code
* @date 2024surname Nian08moon09
*/
@Component
@Slf4j
public class TiebaHotSearchJob {
@Autowired
private SbmyHotSearchService sbmyHotSearchService;
@PostConstruct
public void init() {
try {
// call (programming) hotSearch methodologies
hotSearch(null);
} catch (IOException e) {
("启动时call (programming)热搜爬虫任务异常", e);
}
}
@XxlJob("tiebaHotSearchJob")
public ReturnT<String> hotSearch(String param) throws IOException {
("Internet search engine爬虫任务开始");
try {
String url = "/hottopic/browse/topicList?res_type=1";
List<SbmyHotSearchDO> sbmyHotSearchDOList = ();
Document doc = (url).get();
//caption
Elements titles = (".topic-top-item-desc");
//Hot Links
Elements urls = (".topic-text");
//Hot Search Index
Elements levels = (".topic-num");
for (int i = 0; i < (); i++) {
SbmyHotSearchDO sbmyHotSearchDO = ().hotSearchResource(()).build();
//设置文章caption
((i).text().trim());
//Setting up article links
((i).attr("href"));
//Setting up the Knowledge TripartiteID
(getValueFromUrl((), "topic_id"));
//Setting the heat of a hot search
((i).text().trim().replace("WLive discussion", "") + "ten thousand");
//ordinal order
(i + 1);
(sbmyHotSearchDO);
}
if ((sbmyHotSearchDOList)) {
return ;
}
//Data added to cache
CACHE_MAP.put((), ()
//Hot Search Data
.hotSearchDTOList(
().map(HotSearchConvert::toDTOWhenQuery).collect(()))
//update time
.updateTime(().getTime()).build());
//Data persistence
sbmyHotSearchService.saveCache2DB(sbmyHotSearchDOList);
("Internet search engine爬虫任务结束");
} catch (IOException e) {
("Get posting data exception", e);
}
return ;
}
/**
* Getting parameters from a link
*
* @param url link (on a website)
* @param param The parameter whose value you want to extract
* @return
* @throws Exception
*/
public static String getValueFromUrl(String url, String param) {
if ((url, param)) {
throw new RuntimeException("Getting parameters from a link异常，urlmaybeparamempty");
}
try {
URI uri = new URI(url);
String query = ();
Map<String, String> queryPairs = new HashMap<>();
String[] pairs = ("&");
for (String pair : pairs) {
int idx = ("=");
String key = ((0, idx), StandardCharsets.UTF_8.name());
String value = ((idx + 1), StandardCharsets.UTF_8.name());
(key, value);
}
return (param);
} catch (Exception e) {
("An exception occurred in the extraction parameter", e);
throw new RuntimeException("Getting parameters from a link异常");
}
}
}